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ABSTRACT 



This final report discusses the outcomes of a project that 
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receiving assistance from this project provided services to students in fully 
inclusive general education classrooms, in partially inclusive classrooms, in 
special education classes, in psychoeducational programs, and in day 
treatment community-based settings. The goals of the program were to: (1) 

increase the understanding of educators and families of students with severe 
disabilities about exemplary teaching and behavior management practices for 
facilitating social -emotional competence and responsible behavior; (2) 
increase the skills of educators in selecting, implementing, and 
demonstrating exemplary practices; (3) facilitate the effectiveness of state 
and local education agencies in implementing quality adoption sites and 
programs with personnel skilled in demonstrating exemplary practices; and (4) 
assist state and local agencies in providing technical assistance, 
information dissemination, and personnel development. At the end of the 
3 -year period, the project exceeded anticipated outcomes for each management 
objective. Details of each management activity and its accomplishments are 
provided in the report, as well as recommendations for effective outreach and 
technical assistance. (CR) 
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EXECUTIVE SUMMARY 



DEVELOPMENTAL THERAPY-DEVELOPMENTAL TEACHING: 

An Outreach Project For 

Children With Severe Disabilities (CFDA No. 84.068U) 

October 1, 1996 — September 30, 1999 

Final Report 

This project, Developmental Therapy-Developmental Teaching: An Outreach Project for Children 
and Youth with Severe Disabilities (CFDA No. 84.068U), provided outreach assistance to programs 
serving children and youth, ages 3 to 16 years, with severe social, emotional, and behavioral 
disabilities (and those with other disabilities including autism, when severe problem behavior was also 
a disability). Programs receiving assistance from this project were providing services to these children 
and youth in fully inclusive general education, in partially inclusive general education, in special 
education classes, in psychoeducational programs, and in day treatment community-based settings. 
The original goals remained unchanged during the three years of the project. 



Project Goals 

1. Increase the understanding of educators and families of children and youth with 
severe social, emotional, or behavioral disabilities in early intervention, preschool, 
elementary and middle school and in other community settings, about exemplary 
teaching and behavior management practices for facilitating social-emotional 
competence and responsible behavior. 

2. Increase the skills of educators in selecting, implementing, and demonstrating 
exemplary practices based on their increased understanding of the special program 
needs of these students. 

3. Facilitate the effectiveness of state and local education agencies in implementing 
quality adoption sites and programs with personnel skilled in demonstrating 
exemplary practices which enhance teaching-learning environments for these 
students. 

4. Assist state and local agencies in providing technical assistance, information 
dissemination, and personnel development for coordination and replication of 
model components to meet the needs and requests across the states and nation. 



Project Activities 

Outreach services included dissemination of information about the model (management objective 1); 
consultation and planning for model implementation (management objective 2); inservice training 
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for model implementation through workshops, in-class tutorials and in-depth follow-up 
(management objective 3j; coordination with state and national agencies (management objective 
4); outreach assistance for professional development through workshops, distance learning, and 
teleconferencing (management objective 5); preparation of inservice instructional sequences and 
media for use with local programs (management objective 6); design of new outreach activities, 
including a training-trainers program, and modification existing outreach strategies to meet changing 
needs of personnel in multiple settings (management objective 7); and evaluation of project 
accomplishments in meeting needs of programs and individuals at each site, with particular focus on 
improving the performance and effectiveness of the service providers (management objective 8). 

On a year-by-year basis, the project worked with 9 programs during the first year of the grant; 
10 programs the second year (5 continued plus 5 new programs), and 16 programs in the third project 
year (9 continued plus 7 new programs). Details of each management activity and its 
accomplishments are provided in following sections. 

Project Outcomes 

At the end of the three-year period, the project exceeded anticipated outcomes for each management 
objective. Through dissemination activities, the project reached a documented total of 5,480 
individuals in 46 states. Virgin Islands, and 13 foreign countries seeking information about the model 
and/or outreach assistance. Through conferences and workshops, 1,037 individuals received inservice 
training. Local needs assessment and for planning model implementation was provided to 21 
programs in 8 states. Of these, 18 programs, 179 individuals serving 423 children with special needs, 
received in-depth, extended outreach assistance for model implementation during the three-year 
period. Figure 1 provides an overview of these project accomplishments. 

Project Effectiveness 

Project effectiveness was defined as (a) personnel with demonstrated proficiency in their own service 
settings for facilitating emotionally healthy development of children with severe disabilities, especially 
those with severe social-emotional-behavioral problems, (b) increased social-emotional-behavioral 
competence of these children during staff training for model replication, (c) satisfaction of 
participants with the instructional sequences and training materials, and (d) programs providing 
sufficient resources to support effective model implementation. 
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Figure 1. Overview of Performance Indicators, Final Performance Report, Oct. 1, 1996 • Sept. 30, 1999 



Management Objective 


States Reached 


SchoolsfSites 

Served 


Individuals 

Reached 


Children 

Benefitting 

Directly 


Other Outcomes 


1. DISSEMINATION* 


46 states & 
virgin Islands; 
13 other 
nations 


NA 


5,480 individuals 


NA 


1,903 awareness 
materials distributed to 
1,952 individuals; 1,625 
newsletters mailed; 109 
requests for monographs 


2. PLANNING FOR 
MODEL 

IMPLEMENTATION* 


8 states 


21 programs 


371 individuals 


5,581 children 


21 training agreements 


3. MODEL 

IMPLEMENTATION & 
REPLICATION 


8 states 


18 programs; 
1,512 hours of 
direct on-site 
consultation & 
instruction 


179 individuals 
with extended in- 
depth training 


423 children 
with special 
needs 


16 programs continuing 
with model components 
after training 


4. INTERAGENCY 
COLLABORATION* 


10 states 


30 agencies and 
universities 


NA 


NA 


Additional ftmding 
support received from 
Georgia and 
Washington State 


5. PROFESSIONAL 
DEVELOPMENT* 


National 

outreach 


21 programs 


1 ,037 individuals 
39 workshops 


NA 


24 other professional 
activities 

(presentations/exhibits, 

publications) 


6. PRODUCT 
DEVELOPMENT* 


NA 


NA 


NA 


NA 


Web site; 2 videos 
introducing the model; 
FB A software; internet 
course modules 


7. PILOTING TRAINING 
OF TRAINERS* 


6 states 


28 leadership 
trainees 


28 schools & 
agencies 


NA 


2 leadership trainees; 
completed new 
certification program, 
17 others in process 


8. EVALUATION 
(See Tables 1-11) 


NA 


Model fidelity 
measured at 9 
replication sites 


Sample 

performance data 
analyzed for 79 
direct service 
trainees 


Sample 

performance of 
199 children 
analyzed for 
social/emotional/ 
behavioral gains 


Data storage/retrieval 
system established; 
satisfaction survey fron 
participants; focus group 
feedback from 
leadership trainees 



♦During Project Years 2 & 3 costs and resources were shared with our Early Childhood Outreach Project. 
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Observational ratings of actual performance of a representative sanple of direct service 
providers, indicated that 83% acquired a proficiency score of Adequate or better by demonstrating 
basic practices necessary for model implementation. Of these, 79% achieved higher proficiency scores 
at Effective or Highly Effective levels of proficiency. Among the 28 leadership trainees in the new 
pilot program for training trainers, 82% achieved the passing criterion for in-depth knowledge of the 
model. During the short time this pilot effort was initiated two of these program coordinators were 
able to corrplete proficiency requirements in all five competency areas and receive certification as a 
Developmental Therapy-Developmental Regional Associate. The remaining leadership participants 
continue to pursue their individual plans for acquiring certification. 

Measures of satisfaction of participants with their training experiences indicate that project 
activities met their needs, and most respondents indicated considerable gains in understanding and 
s kills Almost all participants also indicated a need for further training or more time with project 
instructors on-site. Workshop effectiveness, assessed by 815 participants (including participating 
parents) received average ratings of 4.5 to 4.7 on a scale of 5 {Highly Satined) to 1 (Not Satined), 
indicating high degrees of satisfaction. Satisfaction of direct service teams, assessed through post- 
project anonymous questionnaires, indicates levels of satisfaction fi'om above average (ratings >3.0) 
to highly satisfied (ratings of 5.0) on all four project training dimensions: workshops, observations 
in their classrooms, debriefings for feedback, and written feedback. Views of leadership trainees about 
their satisfaction and usefulness of their project experiences was assessed through a focus group 
discussion. The participants held high opinions of their experiences both professionally and personally. 

Measures of social-emotional-behavioral development of a representative group of 199 
children at 9 sites during the model implementation period indicate that the group made statistically 
significant progress (p<. 000), indicating that model implementation activities had a positive effect 
in promoting social-emotional-behavioral development of the children. 

A final measure of effectiveness was obtained by interviewing local coordinators to assess the 
extent to which participating programs acquired the basic elements for model replication. Of the 9 
sites that participated in evaluation of child progress, all were rated at the Basic Implementation level 
or better, and two sites achieved the highest Exemplary Model Demonstration level. 

Together, these evaluation results indicate that the overall project mission to in^rove service 
for children and youth with severe social-emotional-behavioral disabilities was achieved with distinct 
and measurable performance indicators. 

***** 
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FINAL REPORT 

DEVELOPMENTAL THERAPY-DEVELOPMENTAL TEACHING: 

An Outreach Project For 

Children With Severe Disabilities (CFDA No. 84.068U) 

October 1, 1996 — September 30, 1999 

THE INTERVENTION MODEL 

ThQ Developmental Therapy-Teaching curriculumprovides a framework for guiding social-emotional 
development and responsible behavior in children and teens. It matches a child’s current social, 
emotional, and behavioral status with specific goals, objectives, behavior management strategies, 
curriculum materials, activities, and evaluation procedures. It also defines specific roles for adults to 
facilitate a child’s development. The curriculum sequentially spans social, emotional, and behavioral 
development for children and youth from birth to 16 years. 

The ciuriculum has four areas; Behavior, Communication, Socialization, and (Pre) 
AcademicsICognition, to address four essential human activities — doing, saying, caring, and 
thinking. Within each of these four areas, specific teaching objectives follow developmental sequences 
for social-emotional competence and responsible behavior. Specific curriculum activities, management 
strategies, and adult roles define the ways the model is implemented for preschoolers, school-aged 
children, and teens. 

Three measurement instruments provide the core evaluation measures for this curriculum. The 
Developmental Teaching Objectives Rating Form-Revised (DTORF-R) is a 171-item assessment 
instrument used to obtain a profile of a child’s social-emotional-behavioral status. It identifies specific 
objectives for social-emotional conpetence in an Individualized Education Program (lEP), 
Individualized Family Service Plan (IFSP), or Individual Transition Plan (TTP). The rating process 
is used also for a functional behavioral assessment, provides a profile of current strengths as well as 
areas of difficulty, and is used at repeated intervals to evaluate child progress. 

The Developmental Therapy Rating Inventory of Teacher Skills (DTRJTS) has four forms 
specifying the basic adaptations in practices for model implementation in four large age groups: 
infant/toddlers, preschool, elementary school-aged, and in middle/high school. The DTRITS provides 
an observational rating of an adult’s current performance skills, serves as a needs assessment for 



planning inservice training, is the basis for tutorial feedback, can be used as a self-guide for model 
implementation, and documents acquisition and maintenance of skills over time. DTRITS data also 
provide measures of replication fidelity at sites attenpting model implementation. An Administrative 
Support Checklist contains 41 basic administrative elements associated with levels of program quality 
in model replication. Previous studies of model effectiveness have shown that certain minimal levels 
of administrative support were necessary to support successful performance by direct service teams 
in classroom settings as measured by the DTRITS during a school year. 

The evaluation plan uses these three instruments to obtain measures of both qualitative and 
quantitative assessment of outreach activities and the optimal settings/conditions for achieving the 
greatest results. These measures of trainees, children, and programs were analyzed for evaluation of 
outcome effectiveness. The benefits firom such analyses are these: 

• Formative feedback to individual participants re- focuses training so that learning experiences 
can be redefined, reinforced, revised, and replicated. 

• Summative feedback documents project accomplishments and permits staff to examine the 
quality of outcomes. 

HOW THE PROJECT GOALS WERE ACCOMPLISHED 

The Developmental Therapy-Teaching Programs is an outreach unit of the College of Family 
and Consumer Sciences at the University of Georgia, Athens, Georgia. The unit enjoys outstanding 
administrative support and working relationships with the Office of the Vice President for Services 
and Outreach, Dr. Eugene Younts; and in the college, with Dean Sharon Nickols and Associate Dean 
Christine Todd. The unit is comfortably housed off-campus due to a critical space shortage at the 
University, but is able to connect directly to all of the on-campus support systems. Appendix A 
illustrates the administrative organization of the unit within the University. 

During the three years of this grant project, the unit received additional grant support for 
other outreach, training, and service activities firom the Georgia Department of Education, U. S. 
Department of Education Office of Special Education Early Education Programs (CFDA 84.024C), 
State of Washington Department of Social and Health Services Division of Children and Family 
Services, and local public education and community service programs. 

Project staffing went through several changes during the three grant years. The original 
Project Director, Karen R. Davis became ill during the first year and subsequently went on disability 



status and died. Serving as part-time Project Director was Dr. Mary M. Wood, retired professor 
emeritus of special education and founder of the Developmental Therapy-Developmental Teaching 
model. Replacing Davis as senior trainer was Dr. Connie Quirk, an experienced and certified National 
Instructor for Developmental Therapy. Dr. Faye Swindle served as part-time Senior Training 
Associate for the entire three-years of the project. A second Training Associate, Julie Hendrick 
became ill also during the first few months of the project and was retired on disability. Her part-time 
position was filled by Diane Wahlers as Coordinator of Outreach and Distance Learning during the 
second and third grant years. The Coordinator for Dissemination, Betty DeLorme and Office 
Manager Debbie Huth served the project part-time throughout the three grant years. In addition to 
this core staff, the project was able to obtain the services of four highly experienced and certified 
National Instructors in Developmental Therapy as adjunct stafficonsultants to assist with in-depth 
training at selected field sites. These were Dr. Bonnie McCarty, Dr. Susan Galis, Dr. Mary Leiter and 
Rosalie McKenzie, a specialist in services to young children with Autism Spectrum Disorders. In 
addition. Dr. W illiam Swan and Dr. Douglas Flors provided services for project evaluation. 

The success of the project and subsequently its mission lies with the fit between project goals 
and agency needs and commitment. Programs and agencies seeking to improve their services need 
aU available information about resources and options. Keenly aware of this need, the project had 
extensive communications with each potential site during planning phases so that expectations of 
administrators and direct service providers were matched to the outreach assistance as nearly as 
possible. This overarching principle guided project activities, while keeping efforts focused on the 
specific project objectives and outcomes as specified in the original proposal. 

Activities and accomplishments are reported below, according to project management 
objectives. 

Management Objective 1, Dissemination: To design and disseminate information about the model 
and available outreach assistance available through the project. 

Model information was disseminated upon request to 5,480 individuals in 46 states. (The 
original project proposal for this objective was to reach individuals and programs in at least 8 states.) 
Internationally, the project received requests for materials and information from the Virgin Islands, 
Australia, Canada, England, Germany, Indonesia, Israel, Lithuania, New Zealand, Russia, Scotland, 
Singapore, Taiwan, and Ukraine. 
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The disseminated print materials included “awareness” packets of information distributed to 
1,903 individuals; brochures, flyers, and other informational materials to 1,952 individuals; and three 
issues of the newsletter, distributed to 1 ,625 individuals. A fourth newsletter is currently in press for 
distribution during Winter, 2000. Copies of the newsletters are included at the end of this report. 

Additionally, 109 requests were received for two project monographs. The first, a 73-page 
monograph describes the 25-year history of the Developmental Therapy-Teaching model. This 
product was useful for potential sites which needed in-depth information about the development of 
the model, references for related publications, summary of research conducted in a variety of settings 
documenting student progress with this model, acquisition of skills by teachers, and adaptations of 
the model for inclusive and early childhood settings. The second monograph, a 33-page publication. 
Documenting Effectiveness, provides potential users with reports of research documenting model 
effectiveness in inclusive, partially inclusive, and special education settings. It also describes 
effectiveness of the model with children ages 3 to 16 years. The findings support use of the model 
with children who have severe social-emotional-behavioral disabilities and those with other disabilities 
who have additional social-emotional-behavioral disabilities. Also included are observational data of 
teaching teams and college interns rated on performance in demonstrating specified practices for 
model implementation when working directly with children and teens. These data provide measures 
of fidelity in model implementation. 

Telephone and e-mail communications were additional forms of dissemination used 
extensively for exchange of information and consultation. 

During Project Year 2, a redesign of materials was needed to include internet access for 
expanded dissemination and training activities. The project web page, www.uga.edu/dttp. provided 
resources and basic model information. An additional web-based course module for learning to use 
the model s procedure for assessment of a child’s social-emotional-behavioral development was 
developed during Project Year 3. [For additional information on web-based outreach and training, 
see management objective 6.] 

Management Objective 2, Site planning for model implementation: To determine the training 
needs of participants and pro grams and to design outreach services which reflect participant needs 
with ensured follow-up and support activities. 

We called this phase of outreach. Site Development Assistance. Following a preliminary 



request for outreach assistance, the planning focus was to identify the training needs at local program 
sites and the degree of commitment from staff, parents, and administrators to provide necessary 
resources and support for model replication. Criteria for accepting a site for model replication were: 

/ Evidence of administrative support and need for services to be provided for model 
implementation. 

/ Evidence of sufficient staff planning to demonstrate basic knowledge about the model 

and a willingness to attempt model implementation. 

/ One supervisory person from the site who agreed to participate in the Training 
Trainers Program while outreach services are provided to the direct service personnel 
(New requirement, begun in Project Year 2.) 

/ A Training Agreement with content needs and training schedule coUaboratively 
developed from a needs assessment developed by project staff, program 
administrators, direct service providers, and parents. 

/ A student evaluation schedule for the year, including a minimum of pre- post- 
measures to be submitted to the project without student names attached, including the 
Developmental Teaching Objectives Rating Form-Revised (DTORF-R) and other 
evaluation measures routinely used by the program. [See management objective 8, 
Evaluation.] 

/ Agreement to use the DTORF-R for lEP program planning and to provide family 
services and program evaluation consistent with the principles of the model 

/ Commitment of staff time to training with the understanding that periodic 
performance measures would be collected by the project instructor with feedback to 
the participants. 

Figure 2 lists the sites participating in planning for model implementation, showing the year 
a site initiated planning and the extent of carryover of training from year-to-year. As shown, 21 sites 
received planning assistance for model implementation. After the first project year, we received a 
second outreach grant focusing specifically on the needs of very young children with severe social- 
emotional- behavioral problems. With this new project, we transferred outreach services for four early 
childhood programs [sites 6, 7, 8, and 9 in Table 2] where model implementation activities were 



Figure 2. Sites Participating in Planning for Model Implementation by Year (N = 21) 



Year 1. 1996 • 1997 Year 2. 1997 • 1998 

Site Planning Initial Implementation 



1. Fanin & Gilbert County School District, Initial implementation 
Mountainbroc^ Psychoeducational Program 

Blue Ridge, GA 

2. Cherokee County School District, Initial implementation 

Mountainbrook Psychoeducational Program 

Canton, GA 

3. Dalton School District Initial implementation 

Mountainbrook Psychoeducational Program 

Dalton, GA 

4. Cooperative Educational Services* Initial implementation* 

Trumbull, CT 

5. Laurens School District #55 Site planning continued 

Laurens, SC . 

6. Gateway Therapeutic Preschool (CrMitinued through Early Childhood Project) 

Bowmansville, NY 



7. Learning Tree Therapeutic Preschool (CtMitinued through Early Childhood Project) 

Bremerton, WA 



8. Parents Early Intervention Program (Continued through Early Childhood Project) 

Positive Education Program 
Cleveland, OH 



9. Monarch Therapeutic Preschool (Continued through Early Childhood Project) 

Lacy, WA 

Year 2. 1997 - 1998 
Site Planning and 
Initial Implementation 

10. Special Needs* 

Positive Education Program 
Cleveland, OH 

11. Day Treatment (2 locations) 

Positive Education Program 
Cleveland, OH 

12. Warner Robins AFB Schools* 

Warner Robins, GA 

13. Maine School Administrative 
District #40 

Waldoboro, ME 



14. Maine School Administrative 
EHstrict #72 
Fryeburg, ME 
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Year 3. 1998 - 1999 
Replication Level 

Model replication 

Model replication 

Model replication 

Model replication* 
Discontinued 



Year 3. 1998 - 1999 
Replication Level 

Model replication* 

Model replication 

Model replication* 
Partial replication 

Initial implementation 



Figure 2 (continued) 



Year 3, 

Initial Implementatiwi 



15. Cobb’Douglas* 

Psychoeducational Program 
Marietta, GA 

16. Behavioral Health Resources 
Olympia, WA 

17. Ash Street School* 

SouthMetro Psychoeducational Program 
Atlanta, GA 

18. Rat Shoals School* 

SouthMetro Psychoeducational Program 
Atlanta, GA 

Site Planning 

19. Hq3kins G3unty School 
District* 

Madisonville, KY 

20. Presbyterian Child Welfare Agency 
LxHiisville, KY 

21. Child Study Treatment 
Clover Park School District 
Lakewood, WA 



♦Programs marked with an asterisk serve both early childhood and school age children and those with Autism Spectrum Disorders. At these sites, 
outreach assistance was a collaborative effort with our Early Childhood Outreach Project. 
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underway but serving only preschool children. Their data are included in this report for Year 1 only. 
Of the remaining 17 programs during Project Years 2 and 3, 6 programs served both preschool and 
school aged children with Autism Spectrum Disorders or related disabilities, ages 3 to 1 1 years [sites 
identified with an asterisk* in Table 2]. Outreach assistance to these particular programs was shared 
collaboratively with our Early Childhood Outreach Project because the identified needs enconpassed 
a broad developmental and age range. 

Each program differed in staff skills, needs, and resources. Planning for the training sequences 
and implementation processes at each program site was unique to the identified needs. Therefore, 
considerable resources were allocated to the planning phase of outreach, prior to actual training for 
model implementation. 

In planning, needs assessment activities were conducted with potential sites to insure that 
there was sufficient commitment philosophically to provide necessary resources for implementing 
components of the model The planning phase was completed when a formal training agreement was 
negotiated and signed by the program administrator. This agreement formalized mutual agreements 
about (a) amount of training and technical assistance the project would provide, (b) obligations of the 
local site for cost-sharing and released staff time, and (c) scheduled times for repeated evaluations 
of child progress during the training period. Each training agreement specified the implementation 
sequence but could be modified during the process as needed. 

When a local program committed to model implementation, cost sharing was negotiated on 
the basis of the size of the program, the number of participating teams, extent of administrative 
support, and the initial skill levels of participants. Local programs were expected to contribute some 
resources to the effort. We used a minimum cost-sharing approach in which every participating 
program made some degree of commitment, both in cost and in released time for participating 
personnel. The fees were designed to minimize initial costs to local programs during this planning 
phase, with no charges except travel expenses for the project consultant. The project assumed costs 
for training materials and the project consultant. In contrast, there was a rather large proportional 
contribution expected when a program simply requested a single Workshop by Topic [reported in 
management objective 5 below]. We kept this type of assistance to a minimum, except for 
introductory presentations about the model because workshops alone, without follow-up, seem to 
have minimum long term benefits; that is, there is little carryover or skilled implementation. [A copy 
of the fee schedule is included in Appendix B at the end of this report.] 



Management Objective 3, Model Implementation and Replication: To conduct extended in- 
depth training and technical assistance for model implementation at local program sites. 

This management objective received the major portion of project resources and staff time 
because activities involved extended, in-depth training with repeated visits to each participant at each 
site. Overall, during the three project years, activities to inclement the model provided approximately 
1,512 hours of direct, on-site consultation and instruction through inservice, observations, feedback, 
and tutorials at 1 8 program sites with 179 participants working directly with 423 children with special 
needs. (The numbers participating exclude parents and additional local personnel that participated in 
introductory workshops and staff debriefings, but were not in positions to participate for extended 
periods of in-depth training; e.g., parents, social workers, program directors/principals, psychologists, 
general education, teachers. These individuals are described in management objective 5, Professional 
Development.) 

While there was variability in amount of project assistance provided to each program, on 
average a site received 3 visits of three days duration at each visit during a school year [18 contact 
hours per visit x 3 visits = 54 in-depth instructional hours approximately per pro gram per year]. While 
the sites had wide differences in staff experience and skill, they all requested continuing assistance 
after the first year of initial implementation. 

Figure 3 contains a summary of types of programs provided and the age groups served by 
these 18 sites during the three-year period. Of the 21 sites that received assistance in planning for 
implementation, three sites began planning during the latter part of Project Year 3 and did not receive 
more than introductory training for implementation. All three of these new sites are continuing 
implementation activities under different funding sources. 

Typically the implementation phase began with 2 to 3 days of workshops on the core content 
necessary for basic model implementation. Appendix C contains the "content map", developed 



9 




19 



Figure 3. Summary Characteristics of 18 Sites Receiving In-depth Staff Training for Model Replication 



Site 


Number 

Staff 

Trained* 


Type of Program 


Number 

Children 

Served 


Ages 

3-5 

Pre-K 


Grades 

K-3 


Grades 

4-5 


Middle 

School 


High 

School 


Un- 

Graded 


A 


26 


Inclusion 


16 


12 


4 


0 


0 


0 


0 


B 


1- 


InclusicKi 


32 


32 


0 


0 


0 


0 


0 


c 


4 


Special classes 


15 


0 




0 


0 


0 


15 


D 


10 


Special classes 


39 


0 


0 


7 


12 


20 


0 


E 


8 


Partial inclusicxi 


7 


0 


4 


3 


0 


0 


0 


F 


12 


Special classes 


35 


35 


0 


0 


0 


0 


0 


G 


8 


Special classes 


35 


0 


12 


2 


9 


12 


0 


H 


18 


Special classes 


72 


1 


15 


10 


20 


30 


6 


I 


8 


Special classes 


21 


0 


3 


2 


11 


5 


0 


J 


20 


Special classes 


31 


23 


0 


0 


0 


0 


8 


K 


4 


Special classes 


10 


0 


0 


0 


0 


0 


10 


L 


14 


Partial inclusion 


20 


20 


0 


0 


0 


0 


0 


M 


8 


Special classes 


11 


0 


0 


6 


0 


0 


5 


N 


10 


Partial inclusion 


29 


0 


11 


10 


8 


0 


0 


0 


8 


Partial inclusion 
& special classes 


20 


0 


11 


4 


5 


0 


0 


P 


4 


Special classes 


10 


1 


4 


1 


4 


0 


0 


Q 


4 


Special classes 


14 


3 


6 


5 


0 


0 


0 


R 


4 


Special classes 


13 


0 


2 


5 


0 


0 


6 


Total 


179 

trainees 




430 

children 


127 

3-5 yrs 


72 

K-3 gr. 


55 

4-5 gr. 


59 

Middle 

School 


67 

High 

School 


50 

Ungraded 



♦Participating staff worked in teams of 2 persons, minimum. Most programs had additional support staff working in the classrooms who were 
participating in training intermittently. 
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during Project Year 2, outlining the instructional content modules from which specific core sequences 
were selected, based on assessment of training needs. 

A typical training sequence included: 

• Initial workshops and seminars on-site, conducted by the project instructor 
and sometimes co-taught with a leadership person in the Training Trainers 
Program during Years 2 and 3. 

• Observations of participants in their own programs working directly with 
students, followed by team debriefings and feedback about model practices 
they denwnstrated successfully and those skills which needed further 
attention. 

• Video consultation for feedback, when requested by a site, with written 
permission from parents and personnel being video-taped. 

• Periodic observational ratings of participants, performance demonstrating 
their use of model practices followed by team debriefings and 
recommendations for improved practices. 

• Written reports after each site visit reflecting recommendations and 
observations about progress of the site toward achieving the standards 
specified for designation as a replication site. 

Management Objective 4, Interagency Collaboration for Model Replication: To coordinate 
outreach activities with state and national agencies as they strive to improve services for children 
with severe disabilities, and to respond to comprehensive personnel development needs. 

The project collaborated with 4 state agencies responsible for personnel development 
networks to ensure consistency with personnel standards and plans. Correspondence with these 
agencies provided this project with the State plan for personnel development in Georgia and 



assurances from the designated state agencies in Washington, Georgia, Ohio, Kentucky, and Maine 
that the project was consistent with State plans. 

The project was approved by 8 universities in 5 states for staff development units (SDUs). 
The agencies for licensing child care providers approved the Developmental Therapy-Teaching 
Programs training under licensure standards in the State of Washington and in Georgia. Additionally, 
Continuing Education Units (CEU's) were provided to participants in the Black HiUs Seminars in 
South Dakota through Augustana College; in Maine through the School Psychology Department of 
Southern Maine University, in Wisconsin through Silver Lake College Spring Tonic Conference; in 
Texas at the International CCBD Conference through University of North Texas; and in Georgia 
through Georgia State University and the University of Georgia Center for Continuing Education. 

Project staff collaborated with 18 national and state professional associations and agencies 
to assist in planning and/or assisting in implementing the Developmental Therapy-Teaching model. 
These included CEC, CCBD, DEC, NEC*TAS, National Research Institute on Children's Mental 
Health, Head Start program of Audubon Area Community Services, Inc. in Kentucky, the Mental 
Health Center of North Central Alabama; Georgia Public Television; Northeast Georgia Regional 
Educational Services Agency; Georgia Department of Human Resources Early Intervention Program; 
Gateway Youth and Family Services in WiUiamsville, New York; EASTCONN in northeast 
Conneticut; Maine State Billing Services, Inc. for Medicaid planning.; and in the State of Washington 
EPIC in Yakima, Educational Services District #114, Educational Services District #113, and the 
Department of Social and Health Services, Mental Health Division and the Therapeutic Child Care 
Program in the Children's Administration. 

Management Objective 5, Professional Development: To provide topical workshops and other 
forms of professional resources for individuals, local programs, and professional organizations to 
extend outreach, training, and technical assistance in model implementation. 

While this objective originally was intended to provide topical workshops on request for 
professional development through inservice, our experience indicated that the greater benefits for 
professional development can accrue when participants have extended periods of intermittent training 
and follow-up with in-depth tutorial assistance directly in their classrooms. For this reason, single 
workshops and other forms of on-day training were minimized in the project, while extended, 
in-depth training was given a greater project priority [as described above in management objective 



3, Implementation]. 

Topical workshops . Over the three-year period, 1,037 individuals participated in topical 
workshops. (The original proposal anticipated about 350 individuals would be served through 
workshops.) During Project Years 1 and 2, workshops were presented at 8 locations with 172 
participants. About 41% were teachers, 22% were paraprofessional aides/educational technicians, and 
the remainder were coordinators/supervisors (9%), family service providers (8%), program 
administrators (6%),psychologists (5%), social workers (5%), speech/language therapists (2%), and 
others (4%). Many of these participants later became part of direct service teams at irrplementation 
sites [See management objective 3 above.] The topics requested typically included a day-long 
overview of the model and its basic components: Behavior Management, Assessment of Students' 
Social-Emotional-Behavioral Development, Roles of Lead and Support Teachers, Building a 
Therapeutic Team, Functional Behavioral Assessments, Group Dynamics and Social Roles in 
Groups, Creative Arts for Therapeutic Gains, and Curriculum Practices that Address Emotional 
Needs. 

During Project Year 3, workshops and other forms of professional development activities 
expanded with a greater emphasis on actively including parents and foster parents in the training. In 
addition, as the new leadership trainees began to co-teach with project staff at sites where model 
implementation was beginning, the project was able to expand the number of sites receiving planning 
and implementation assistance. Between September 1998 and September 1999 in the last project year, 
31 workshops were conducted for 865 participants (with 643 responding to the evaluation 
questionnaires). About 37% of the respondents were teachers, 16% were para-educators or 
educational technicians, and the remainder were administrators, coordinators/supervisors (14%), 
family service providers (8%), parents (6%), social workers (4%), psychologists (3%), and others 
(12%) including art, occupational, physical, educational, and behavioral specialists, students, bus 
drivers, nurses, mental health technicians and others not specified. The proportions and varieties of 
occupations represented at these professional development sessions seem to reflect changing trends 
toward a much more diverse group of individuals working with troubled children and youth. 

Other professional activities . Project staff presentations, writings, collaboration, training and 
exhibits related to model dissemination and implementation for professional organizations, 
conferences, and seminars included the following: 
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Black Hills Seminars, Reclaiming Children and Youth, South Dakota, June, 1997 
Keynote address, Washington State Re-Ed Conference, August, 1997 
CCBD International Conference, Denton, TX, Oct., 1997 

Collaborative research report and publication of pilot program in Winnipeg, Canada, Oct., 1997 

OSEP Project Directors Meetings Jan., 1998, Washington: presentation "Using Evaluation to Build 
an Effective Outreach Program" (over 35 individuals) 

Teleconferencing seminar with University of North Texas doctoral students - Feb. 17, 1998 

Pre-conference and conference workshops. Black Hills Seminars, Reclaiming Children and Youth, 
"Voices for Change", South Dakota, June, 1998 

Seminar, Healing Racism, Star Commonwealth, Michigan, Aug., 1998 

Audio-taped lecture for University of Missouri-Columbia project, Nov., 1998 

CEC Annual International Convention, Minneapolis: poster session "Managing the Ever-Changing 
Pulse of Classroom Dynamics" (over 90 individuals) April 15 - 18, 1998 

Exhibit at conference on Autism: Enriching the Quality of Life, Emory University, Atlanta: June, 
1998 (over 50 individuals) 

Developmental Therapy-Teaching Leadership Conference, Athens, Georgia, April 1998 (22 
participants). 

Pre-conference and conference workshops. Black Hills Seminars, Reclaiming Children and Youth, 
Spearfish, South Dakota, June 1999 

National Technical Assistance Center for Children's Mental Health, June 1998, 
Orlando:NEC*TAS-OSEP panel "Early Childhood Systems of Care" (over 50 individuals) 

Interview for doctoral dissertation at University of North Texas, June, 1999 

National Educational Service manuscript review, August 1999. 

Planning with Buffalo State College for pre-service personnel preparation project. Sept., 1999. 

Audio recording of lecture, "The Developmental Perspective", for University of Missouri-Columbia 
and Arkansas State University project. Enhancing Teacher Problem Solving skills in Early Childhood 
Behavioral Disorders, Nov., 1998 

Silver Lake College Spring Tonic Conference, March 1999, Wisconsin: University students & faculty 



Northeast Georgia Regional Educational Services, April, 1999, Athens, GA: Meeting the educational 
needs of students with severe emotional/behavioral disabilities. 

CCBD International Conference, Sept.,-Oct. 1999, Austin, Texas: day-long, four-session strand on 
current and future directions for psychoeducation. 

1999 CCBD Mini-Library Series monograph. Psychoeducation: An Idea Whose Time Has Come, 
with Larry Brendtro, Frank Fecser, and Polly Nichols 

Among experienced program administrators, supervisors, and coordinators there were 
distinctly different kinds of professional development needs. With staff turnover annually, local 
leadership individuals find themselves in need of providing introductory level inservice training for 
new personnel while simultaneously offering advanced skill training for experienced staff. They also 
indicated a need for assistance with their own supervisory and program consultation skills. For this 
group, we initiated a pilot training-trainers program. Regional Associates (RAs) for Developmental 
Therapy-Developmental Teaching during Project Year 2. [This program is described below in 
management objective 7.] 

Management Objective 6, Product Development:. To develop new materials and revise existing 
training materials for greater ^ectiveness in meeting training needs of local service providers for 
model implementation in a changing educational climate. 

In order to reflect current needs and trends of the field, project staff worked continually to 
develop, disseminate, evaluate, and redesign the extensive products, materials, and instructional 
modules used in this outreach project. These new materials/products included (a) new video 
productions for introduction to the model, (b) computer-aided materials for meeting 1997 IDEA 
requirements for functional behavioral assessments, (c) new training materials for trainers-in-training, 
and (d) new training materials for skill practice by site personnel. These products are described below: 

New video productions . In addition to new printed awareness materials and portfolios of 
model information, the project completed a series of introductory 20- minute videos in collaboration 
with the University of Georgia Center for Continuing Education, Media Production Department. (The 
original proposal projected a series of four videos.) The videos introduced principles of the 
Developmental Therapy-Developmental Teaching approach and illustrated exemplary practices to 
demonstrate developmentally appropriate strategies and environments which encourage social- 
emotional growth. These videos were designed for personnel and families of children in three age 



groups; (a) Early Childhood and ECSE programs, (b) those in elementary school programs, and (c) 
those in middle and high school programs. The first, “Providing Developmental Therapy-Teaching 
Programs for Little Ones”, was conpleted in March, 1998. The second, “Developmental Therapy- 
Developmental Teaching for Troubled Children in Elementary School”, was completed in August, 
1999. The third was re-designed into an interactive format with a newer technology - a CD-ROM. 
Additional funding was needed to acconplish this and was received from the U. S. Department of 
Education Office of Special Education Programs. This project in currently underway under separate 
funding. The fourth proposed video in this series, for training personnel to use the Developmental 
Therapy-Teaching Objectives and Rating Form-Revised (DTORF-R) with reliability, was also re- 
designed into newer technology using an internet course format, and is expected to be on-line before 
the end of 1999. 

Computer-aided materials. The most recently completed material is a software version of the 
DTORF-R for use by participating sites. This program, available for either MAC or PC, enables a 
rating team to generate their results in printed format to attach to their local lEP forms. In response 
to the 1997 IDEA requirements for identifying specific social-emotional-behavioral objectives, (a) 
this software version provides a measure of current level of function, (b) identifies specific program 
objectives, (c) projects anticipated gains, (d) specifies positive management strategies based on the 
lEP, and (e) records repeated assessments periodically to document extent of child progress. 

New training materials for trainers-in- training. In keeping with the increased focus on training 
trainers [the RA leadership program described in management objective 7 below] we cortpiled core 
instructional units most frequently requested by sites. These structured “lessons” were requested by 
the leadership trainees as a means to assist them as they began to conduct inservice workshops and 
training independently. With new grant funding the first draft is being cortpiled, field-tested, and 
revised as a Manual for Instructors in Developmental Therapy-Teaching. 

New traininp materials for skill practice bv site personnel. In addition to redesigning handouts 
and participant exercises to support existing training workshop modules during Project Year 2, we 
began to reshape this management objective to focus on distance learning and web-based 
communication. A strategic plan was developed for the expanded use of technology and distance 
learning through teleconferencing, consultation and training via the University satellite system, and 
the internet. [See Appendix D.] . 

During the last year of the grant, many instructional modules were revised as a result of 



evaluation feedback from participants at the local sites. In addition, efforts were nnade to obtain 
additional funding for continuation of our development of new technology — specifically interactive 
internet course work and CD-ROM insfruction, offering useful tools for adults to learn and practice 
at their own learning rate. 

Management Objective 7, Pilot Program for Training Trainers: To design new outreach 
activities to meet expressed needs of personnel in multiple settings. 

In Project Year 2, we instituted a pilot program to field test a training- trainers program for 
leadership individuals to prepare them to (a) conduct awareness sessions and basic inservice training 
for new staff to use the basic model conponents, (b) guide their experienced staff in maintaining high 
quality performance, and (c) assist new local programs in planning and inplementing the model In 
this preliminary pilot effort we received 38 applications during Project Year 2 and accepted 28 
individual in 6 states who met the prerequisite requirements. Not all were expected to conplete this 
pilot training during the project funding period, and other funding sources were obtained to enable 
these initial participants to continue their training for certification as a Regional Associate (RA) for 
Developmental Therapy-Developmental Teaching. 

Criteria for acceptance into the RA training- trainers program were: 

^ Hold a current supervisory/coordinating position with responsibilities for direct 
supervision in a program planning to inplement the model 

^ Submit a resume of prior experience related to work with children and teens who have 

severe social-emotional-behavioral disabilities. 

^ Provide a letter of recommendation about current work from a supervisor. 

^ Complete a preliminary needs assessment. 

^ Complete a pre-assessment test of basic knowledge about the model and how its 
theory is translated into "best practices" for the students to be served. 

Commit a follow-up with model outreach activities independently for at least one new 
site a year following certification. 

Those accepted into the RA training -trainers program began by planning their individual 
training programs with a project instructor so that their training occurred simultaneously with co- 
teaching/supervision, of local program personnel began. Appendix E contains the names and 



positions of these leadership individuals. 



First year training requirements for participating RAs were these: 

/ Complete on-line independent study modules about the basics of the Developmental 
Therapy-Teaching model and the underlying principles. 

/ Participate and co-teach with the project instructor for on-site workshops. 

/ Observe with the project instructor as direct service teams learn to inclement the 
model in their work with students. 

/ Complete three observational ratings of the teams with the instructor: a baseline rating 

at the 1** observation and two practice ratings diudng the 2“^ & 3^** observations. 

/ Debrief with the instructor and the observed team following each observation to 
provide feedback about ways they are demonstrating model practices and ways to 
improve their performance. 



Second year RA requirements included: 

/ Plan and implement (with instructor's assistance) introductory workshops for new 
staff nnembers and additional workshops on essential model elements for all returning 
personnel 

/ Observe teams with the project instructor during all site visits. 

/ Complete three observational ratings with the instructor: two practice ratings with 
feedback (r‘ & I”'* visits) and one "reliability" rating to measure degree of agreement 
with the instructor, item-by-item. 

/ Debrief with the project instructor and teams after each observation about ways they 
are demonstrating model practices and ways to improve their performance. 

/ Assist the project instructor in preparing written descriptions of workshop activities 
proven effective in staff development for implernenting the model 

/ Achieve specified performance standards in each competency area (see Appendix F.) 



The workload and resources for this pilot effort were shared coUaboratively with our new 
Early Childhood Outreach Project. Grant support for an additional project has also been received to 
expand and refine the RA program during the next three years; thus enabling RAs to conqjlete their 
certification requirements and extend their own outreach activities. At the end of the final project 
year, two participants had conqjleted all certification requirements, and seventeen others were actively 
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nearing conq)letion of the requirenients. (For details, see managerrient objective 8, Evaluation.) 



Management Objective 8, Evaluation; To evaluate project effectiveness in meeting the original 
project goals on time and within budget. 

Acconq)lishments for each management objective were evaluated for timeliness and 
effectiveness. Figure 1, presented at the beginning of this report, summarized the scope of project 
acconq)lishments. Forms and instruments have been developed and field-tested in previous projects. 
They are included in the original proposals with descriptions of their development, reliability, validity, 
and uses. It is particularly important to note that the original reliability and validity studies about these 
instruments and subsequent research reports using these instruments were submitted to U.S. 
Department of Education, Program Effectiveness Panel. These studies were conducted with 
populations of students with identified social, emotional and behavioral disabilities. The performance 
measures in these studies resulted in documentation of program effectiveness and model validation 
three times - first as an effective model for children with severe social-emotional-behavioral 
disabilities; second, as an effective training program to increase performance competencies of those 
who work with these students; and third, as an effective model for use in inclusive, partial, or special 
education settings. 

Simple descriptive statistics and written reports were used to summarize frequency data, 
describe extent of effort, and characterize the several groups of participants and service settings. [The 
evaluation plan, procedures for data collection, and measurement instruments were described in the 
original proposal. Evaluation Plan, p.37, and wiU not be repeated here. However, the Summary of 
Evaluation Plan from the original proposal, is reproduced in Appendix G for ease of reference.] 

Timeliness was judged by on-going process evaluation activities described in each 
management objective. Overall, the project maintained the work schedule and budget for activities 
and accomplishments as anticipated in the original proposal Changes in key personnel that resulted 
from illness and death during the first project year somewhat slowed the initial accomplishments in 
Project Year 1. However, the targeted activities and acconq)lishments were recovered and exceeded 
during the remaining two years. 

Effectiveness of the project was assessed on four dimensions: (a) observational measures of 
participants’ performance in using the specified practices; (b) progress of the children served by the 
participants during model implementation; (c) satisfaction of the participants with the training; and 



(d) assessment of administrative support for model implementation. Tables 1 • 11 located at the end 

of this report contain these effectiveness measures. 

Reliable data collection at on-going service program sites is a well-documented challenge. 
It proved to be so for this project as well. In order to assure confidence in reliability of the data and 
in the accuracy of the findings, we had to accept smaller numbers in our sarr^les. While this approach 
introduced a question of bias into the selection process, we chose sarr^les which had reliable data, 
excluding those where data were incon^lete or inaccurately collected. We believe the smaller sarr^les 
are representative of the typical participants, children served, sites, and outcomes. 

Evaluation question 1 . Do participants demonstrate understanding of the key roles of adults, 
as defined by the Developmental Therapy-Developmental Teaching model, for fostering healthy 
social-emotional-behavioral development of children and youth? 

The original evaluation plan involved administering a 50-item multiple choice knowledge test 
to answer this question. A decision was made at the end of the first year of the project that a 
knowledge testper se was not entirely suitable because of the intense focus of our technical assistance 
on actual performance and demonstrated skills of participants. The proposed test put some direct 
service participants at a distinct disadvantage and resulted in low scores when their actual 
performance demonstrated understanding of the content. We beUeve that effective performance 
requires understanding of the knowledge base. Therefore, observations of performance using the 
Developmental Therapy-Teaching Rating Inventory of Teacher Skills (DTRITS) was accepted as a 
sufficient proxy of knowledge on the part of the direct service teams. A representative san^le of 
participants performance scores when working directly with children is presented below in evaluation 
question 2. These results show that 83% achieved DTRITS proficiency scores of adequate or better, 
indicating a working knowledge of basic practices needed for model implementation. 

To further evaluate participants understanding of adult roles in model implementation, a group 
of 13 program directors and coordinators volunteered via teleconferencing to participate in a focus 
group to discuss their views of the training program. The in-depth, open-ended interview was 
conducted by the Project Evaluator and the Coordinator for Distance Learning in a one-hour, 
informal conversational format using video tape to record responses. Focus group guidelines 
suggested by M. Q. Patton in How to Use Qualitative Methods in Evaluation were followed for 
conducting the interview and analyzing responses. Here is a quote firom one participant in the focus 
group that reflects their views about successful implementation: 
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Our staff are much more accountable in identifying their adult roles in 
working with children. As opposed to "Why is this kid doing this? 

What is wrong with them?" [they now ask] "What am I doing? What 
can I do differently?" - those types of questions. So there is more 
accountability. There is this magic with DT that is really quite exciting 
when they ask, "What can 1 try?" - we decode and identify what is 
going on and come up with a solution. Then the staff person comes 
back and says "Oh my God! It worked! It was so easy!" - with a lot 
of these kids who have an abandonment issue we start meeting that 
need, then the child stops doing what was so highly destructive and 
disturbing in the classroom. 

Evaluation question 2. Do participants demonstrate effective performance skills in the service 
setting after participation in the training program to implement the model? 

Performance of direct service participants. Model inplementation requires major emphasis 
on close teamwork among the direct service providers and the support/resource staff. When project 
staff made site visits for in-depth follow-up training, an attempt was made to observe all participating 
teams for a minimum of one hour in each classroonx Typically, an observation was then followed by 
a 30 - 60 minute debriefing for feedback with the team, focusing on skills and areas of performance 
that required improvement. At the time irrplementation activities began at a site, the project instructor 
observed each team to obtain a baseline DTRITS rating (Time 1). After the initial implementation 
activities were corrpleted, DTRITS ratings were repeated (Time 2). This procedure was repeated 
each year that the site participated with the project in implementation. 

Table 1 reports the DTRITS scores achieved by 28 teams (79 individuals) at 9 representative 
sites after initial model inplementation, performance feedback, and tutorial assistance. Levels of 
proficiency established for DTRITS scores in previous studies are 90-100 = Highly Effective, 70-89 
= Effective, 50-69 = Adequate, 30-49 = Less than Adequate, and 16-29 = Poor. The scores 
indicate that 22 teams (66 individuals, 83%) achieved DTRITS proficiency scores at the Adequate 
or better level, indicating demonstration of the basic practices necessary for model implementation. 
Of these, 16 teams (52 individuals, 79%) demonstrated Effective or Highly Effective skills. 

Performance of leadership participants in the training-trainers pilot program. An expanded 
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evaluation design was added in Project Year 2 when the new pilot program was initiated for 
leadership individuals in the Regional Associate (RA) Training Program. Of the 28 people meeting 
the requirements and accepted into the program, 2 participants successfully conpleted all 
requirements in the five competency areas and received certification during the two-year pilot 
program. The pilot program was so well received by these leadership participants that 22 self-selected 
to continue active training under a new project, while 4 are currently inactive but indicate then- 
interest in continuing in the future. 

Table 2 summarizes the progress of these RA leadership participants toward achievement of 
the specified standards for each of the five competency areas. Results of the evaluation activities for 
this program component are summarized here: 

Competency 1. Knowledge: The 50-item multiple-choice test of knowledge about 
Developmental Therapy-Developmental Teaching was taken by aU but one of the RAs at the 
beginning of their training. Post-training knowledge tests were administered on an individual basis, 
when RAs requested the test after periods of self-study or as they came to the end of their 
individualized leadership training program Of the 28 RAs, 1 1 requested taking the post-training 
knowledge test as they neared the end of their programs, and 9 (82%) achieved the passing criterion 
or greater. Table 3 reports the pre-and post-training scores for these 1 1 individuals, ranging from 73 
to 86. Note that 2 persons [#12 &17], with past experience using the model, achieved the criterion 
score on the knowledge test at the pre-training administration. The remaining RAs continue then- 
independent progress toward certification as & Developmental Therapy-Developmental Teaching 
Regional Associate. 

Competency 2. Reliability in using the 171 -itemDTORF-R assessment procedure: Leadership 
participants were expected to participate in team assessments of children in their programs and to 
review all DTORF-R ratings for accuracy. This procedures is a quality check on reliability of the 
assessment and requires extra proficiency in the use of the instrument on the part of the RA. Each 
rating was then reviewed by the project instructor for accuracy in the rating procedure and reliability 
of rater judgments. The instructor identified problem areas or inaccuracies in the rating procedure and 
provided feedback to the RA and the rating team When DTORF-R ratings at a site were accepted 
as reliable and valid measures by the instructor, the RA was judged to have passed conpetency 2, 
DTORF-R reliability. Using this procedure, to date 1 1 of the 28 RAs received a “pass”, indicating 
competency in supervising team ratings of social-emotional-behavioral development. 



Competency 3. Reliability in using the 212-item DTRITS observational rating form: RAs 
were expected to observe with the project instructor as teaching teams worked directly with groups 
of children during inplementation of model practices. These parallel observations were made during 
each return visit of the project instructor, and practice DTRITS ratings were made independently by 
the RA and the instructor. Follow-up discussion of rating differences on particular items following 
an observation served as tutorials for the RAs. This procedure was repeated with each visit until the 
DTRITS rating by a RA reached 80% agreement with the project instructor. Using this procedure, 
4 RAs reached the performance criterion to date. [There is a further discussion below about the 
difficulties of leadership individuals in freeing an uninterrupted hour to complete an observation and 
DTRITS rating.] 

Competency 4. Field supervision: Each RA was expected to provide on-going inservice 
assistance to their staff for model inplementation during the periods between project instructors’ 
visits. At the conclusion of the training agreement, or at the time when the RA and project instructor 
believe that implementation reached an acceptable replication level, the teams were asked to 
anonymously rate the quality and effectiveness of the RA in assisting them in effective implementation. 
Using this procedure, 4 RAs conpleted the requirement successfully by receiving average ratings of 
4 or better on an 8-item form with a 5-point rating scale. As this outreach project ends, 13 others are 
actively in process of guiding their program staff in model inplementation and 6 are inactive but 
indicate interest in continuing to use model components. 

Competency 5. Group instruction in basic model elements: Three phases of training were 
used to assist the RAs in developing effective skills for leading staff workshops for model 
implementation. The first phase, completed by 20 of the RAs, involved co-teaching with a project 
instructor in which planning was a combined effort between the instructor and RA. [These sessions 
were evaluated by workshop participants and are reported below in Participant Satisfaction.] The 
second phase required independent presentations when there was no co-teaching but the project 
instructor assisted the RA in planning, selecting strategies, and designing effective workshop 
materials. This second phase was completed by 17 RAs [also evaluated by the workshop participants] . 
The third phase for certification was successfully accomplished by 9 RAs, in which they independently 
planned all aspects of the workshop, led the session, and were evaluated by a project instructor on 
an 18-item rating form with a 5-point scale of effectiveness as a session leader. 

Evaluation Question 3. To what extent are participants satisfied with their training 



experiences? 

Satisfaction with workshops. Table 4 summarizes workshop evaluations from 172 
participants at 8 locations during Project Years 1 and 2 for each of four questions. [It should be 
noted that not all of the participants at these workshops conpleted the anonymous evaluation forms.] 
Using a scale from 5 {Highly Satined) to 1 {Not Satined), the respondents indicated high degrees 
of satisfaction, with average ratings ranging from 4.69 to 4.73. Table 5 contains consistently similar 
evaluations from 23 workshops conducted during Project Year 3 for 643 respondents. All but 9 of 
these workshops were co-taught by project staff with leadership participants in the RA training- 
trainers program. Using the same evaluation form, respondents expressed similar levels of satisfaction 
with the material, workshop organization, general impression of the workshops, and the extent to 
which their individual needs were met. These average ratings ranged from 4.5 1 to 4.74. 

Satisfaction of leadership trainees (RAs). A focus group interview with 13 program directors 
and coordinators in the training trainers program was held around two general topics with sbc specific 
questions. The first discussion focused on the training that they had received, and the second topic 
concerned their perceptions of the training they were able to provide others. In general, the focus 
group held high opinions of their individual training experiences, both personally and professionally. 
Appendix H contains the questions and summary of responses. They identified experiences they 
valued the most (questions 1.1 & 1.2), citing the 3-day leadership retreat for in-depth immersion on 
the model, participating in presentations with project co-instructors, and observations of instructors 
as they provided consultation and feedback in classrooms. The group identified many new skills they 
had acquired (question 1.3), including understanding of the model and how to apply it in different 
situations, working with families to support children’s healthy development, doing assessments, 
decoding feelings, and helping teachers and parents develop effective plans. 

The group was less similar in their perceptions of their own professional and personal 
experiences during training (question 1.4). Several described their self-development as highly 
satisfying and exciting, while others expressed feeling pressure to perform at levels of difficulty 
resulting in feelings of inadequacy. (One individual viewed this pressure experience “unprofessionaUy 
handled”.) Their observations of their effect on children, families, staff, and programs (question 1.5) 
were aU highly positive. 

Their recommendations for design of the leadership program (question 1.6) reflected 
satisfaction with effective aspects such as the notebooks of materials from others’ training efforts, the 



self-evaluation prescreening process to identify individual strengths and needs, opportunities to focus 
on aspects relevant to their daily work, and the system that responded to different learning needs. 

Their recommendations included greater assistance with presentations, opportunities to 
practice presentations with peers for feedback, increased diversity of participating RAs, and increased 
time needed for preparation of presentations. They expressed some disappointment in the value of 
self-help/peer study groups where they attenrpted to learn from each other. 

In discussion about the training they provided others (question 2.1), they were positive and 
confident of their present level of skill for supporting others in schools, consultation and informal 
training with parents, presenting workshops and training new staff in introductory and intermediate 
levels of model implementation, using the model for FB A and Positive Behavioral Intervention Plans, 
and informally supporting staff in consultation about individual children’s needs. They generally felt 
that their work with the project and with their staff (questions 2.2 & 2.3) was well received but 
expressed concern that presentations offered only at the basic level fail to meet the needs of advanced 
participants. 

Their plans for training others in the future (question 2.4) included foster parent training, 
continuing on-site staff training, consultation with other school districts, and training in positive 
behavior management for general and special educators, administrators, and mental health personnel 
They had numerous future project plans using the model. These included finding grant funds for 
expanding the scope of their program’s model in^lementation, using the assessment instruments at 
a statewide level, extending the model into regional school districts, and expanding the curriculum 
resources for the model. 

They all also anticipated continuing their individual tutorial programs and completing 
certification requirements so that they would be able to train others in the future. They were articulate 
about their own strengths and weaknesses and were able to suggest very specific ways in which the 
project could assist them further in gaining the skills they needed (question 2.5). They requested 
project instructors to continue visiting and monitoring their activities and programs; assistance in 
setting up grant-funded pilot programs; assistance in obtaining resource materials, audio-visual aids 
in training for model implementation, and on-line training materials. They also requested an annual 
leadership conference bringing together RAs from across the country for in-depth immersion in 
leadership issues. 

Satisfaction of team participants in training for model implementation . To obtain information 
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about the level of satisfaction among those who participated in on-site training for model 
implementation, a one-page questionnaire was mailed to 97 participants at 7 of the 9 sites agreeing 
to participate in project evaluation activities. Everyone at a site who had signed an attendance sheet 
at group training sessions received the questionnaire, including support staff, administrators, and 
direct service team members. Using a scale from Very Helpful (5) to Hot Helpful (1), they were 
asked to rate four training activities anonymously: workshops, observations in their classrooms, team 
debriefings for feedback, and written feedback. The questionnaire also contained 5 additional open- 
ended questions about their perceptions of skills they had acquired as a result of training, positive 
aspects and weaknesses of the training they received, changes in their effect on children and families, 
and recommendations for future training. 

Responses were received from 24 %. Table 6 summarizes their ratings, indicating levels of 
satisfaction ranging from above average (ratings > 3.0) to highly satisfied (ratings of 5.0). Their 
responses to the open-ended questions reflected a wide range of individual differences in levels of 
training they received, from participants on direct service teams working year-long for model 
implementation to individuals such as support staff who had attended only the introductory staff 
developn^nt sessions. Responses to the question of newly acquired skills included understanding the 
social-emotional development of children better, skill in using the DTORF-R assessment instrument 
and applying the results in direct programming, using behavior management strategies effectively, 
working with parents and staff more effectively, and applying the model in adolescent programming. 

Training activities they cited as strengths included all of the specific content areas in the core 
training. They reported satisfaction with the organization of these sessions, role play opportunities, 
relevant examples, small group exercises, and opportunities to discuss individual cases. They also 
mentioned knowledge levels, helpfulness, support, and skills of project instructors in providing 
practical applications as strengths in the training. 

Their reports of weaknesses of the training focused almost entirely on issues of time. They 
felt a shortage in number of scheduled observations and feedback they received (time limitations on 
the part of the visiting instructor’s schedule) and training after school when they were tired or 
scheduled to leave. They also reflected that they would have liked more direct suggestions for 
activities, curriculum ideas, foUo w-up case studies, and applications in the classroom for writing goals 
and objectives. One respondent expressed concern about the extent to which the DTORF-R met 97 
IDEA requirements for alternate assessment and adequate goals. There was also a comment that the 



comprehensiveness of the curriculum requires in-depth work beyond what can be done given all the 
other daily requirements. 

In response to the question about positive changes in their effect on children and families as 
a result of the training, all but 3 respondents indicated YES, giving specific examples such as a 
common language among staff and families, changes in parents’ awareness of their child’s strengths 
and problems, increased ability to manage problem behavior more effectively, and increased staff team 
work. 

Respondents made many helpful suggestions for our future training of others. Most 
suggestions reflected the need for nwre extended training during the school year and more time with 
the project instructor in their classrooms in focused training as they begin implementation. They 
reflected on the need for a slow, in-depth approach to assimilating the complex aspects of the model, 
and a need to have more in-class support during implementation. For group sessions, they suggested 
repeating basic constructs at each visit [to accommodate new staff and those who may have missed 
out on previous training sessions]. A suggestion was made that more background information of a 
general nature be provided prior to on-site training. They also suggested that group sessions be 
conducted on teacher planning days rather than after school. These detailed responses are included 
in Appendix I. 

Evaluation question 4. Did children show significant progress in social-emotional-behavioral 
competence during the model implementation period? 

To evaluate project impact on children served by the participants during model implementation, 9 
sites agreed to assist us in collecting basic descriptive data and reliable assessments of social- 
emotional-behavioral status using the DTORF-R. All children served by the participants at these sites 
were included if their DTORF-R ratings for social-emotional-behavioral development were completed 
with accuracy and there was at least one repeated assessment at least 2 months but no greater than 
11 months apart. There were 199 children in the sample. 

Table 7 summarizes the characteristics of these children at each site at the time model 
implementation was initiated. They ranged in age fi-om 2 months to 216 months (18 years), with an 
average age of 10.5 years. Boys comprised 85% of the sample and 74% were white. All had at least 
one recorded disability, 68% with a primary diagnosis of severe emotional/behavioral disability, 22% 
with a diagnosis of Autism Spectrum Disorders, and 34% with additional secondary disabilities. Most 



of these children (78%) were being served in intensive special education programs, 2% were in full 
inclusion programs and 20% in partial inclusion. The severity of their disabilities, calculated from the 
extent of delay on their baseline DTORF-R ratings of social-emotional-behavioral development, 
ranged from 10% as severe, 56% as moderate, 30% mild, and 4% were in the range comparable to 
their age peers. 

On average, the time lag from the initial baseline DTORF-R rating (Time 1) to a second 
DTORF-R rating (Time 2) was 7.8 months — approximately one school year. Statistical analysis of 
gains made by these 199 children using paired sample t-tests with a probability level of .05 and a 2- 
tailed test indicate that the group made significant progress (p < .000) during the time when their 
teachers received training for model implementation. 

Analyses repeated for each site indicate that similar statistically significant gains (p < .000) 
were made by the children at 7 of the 9 sites. Table 8 contains these mean DTORF-R ratings, 
standard deviations, standard errors, and months from baseline to the second rating, and Table 9 
contains results of the t-tests of gains. These findings indicate that the children made significant 
social-emotional-behavioral progress during the time of model implementation by participating teams. 

To explore the extent to which implementation of the model may have contributed 
to these gains, comparisons were also made between the actual DTORF-R scores achieved 
and extrapolated scores assuming no intervention with this model. Extrapolated scores were obtained 
by calculating the average rate of item mastery prior to baseline (baseline DTORF-R score -r CA 
months at baseline = prior monthly rate of item mastery) times months in intervention to Time 2. 
Table 10 summarizes these results. From baseline to Time 2, the 197 children made significantly 
greater gains during initial model implementation(average of 7.8 months) than could have been 
achieved had they progressed at their previous mastery rate prior to implementation. Similar findings 
occurred for 52 of these children who continued their programs during the subsequent 6.2 months 
and received a repeated assessment at Time 3. These findings indicate that model implementation by 
the participating teams had a positive effect in promoting increased social-emotional-behavioral 
development of the children that they served. 



Evaluation question 5. To what extent did local programs at participating sites acquire the 
basic elements specified for model replication? 

An administrative checklist containing 41 basic program elements desirable for effective model 
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replication was used by project instructors and site administrators/coordinators to determine the 
extent to which model con^onents had been included in the implementation effort. If a conponent 
was rated as provided and being used consistently, the item was marked YES. If it was used 
inconsistently, the item was marked PARTIAL, and if it was not available or not implemented, it was 
marked NO. The total items marked YES provided an administrative support score for a site. 
Criterion levels established in previous research studies on model effectiveness are these: 26-41 items 
= Exemplary Model Demonstration (components consistently and effectively replicated); 16-25 items 
= Model Adoption (sufficient number of elements to consider model implementation achieved); and 
10-15 items = Basic Implementation (indicating essential components were utilized). Table 11 
reports the administrative support scores at each of the 9 implementation sites that participated in 
child progress data collection for the project. All of these sites were rated at the Basic 
Implementation level or better; indicating administrative planning that provided the essential elements 
for model implementation. Five sites were rated at the Model Adoption level, and two sites achieved 
the Exemplary Model Demonstration level. 

Summary of Evaluation Results 

The preceding review of evaluation data and outcomes indicates that each of the four project 
goals was effectively accomplished and exceeded anticipated outcomes in the original 
proposal. Outcome measures indicate: 

❖ Increased understanding about how to promote healthy social-emotional-behavioral 
growth through exerr^lary teaching and behavior management practices among those 
who work with children and youth who have severe social-emotional-behavioral 
disabilities (SE/BD). 

❖ Increased skill in using practices proven effective in enhancing teaching-learning 
environments for these students. 

❖ Collaborative planning with state, regional, and local service providers to implement 
model programs for children and youth who have severe social-emotional-behavioral 
disabilities. 
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❖ Increased technical assistance, information dissemination and professional 
development opportunities for those who provide education and mental health 
services to these students, their teachers, and their families. 

Together, these evaluation activities support the conclusion that the overall project mission 
to improve services for children and youth with SE/BD was achieved. 

PROBLEMS ENCOUNTERED AND HOW THEY WERE SOLVED 
There were several unanticipated problems which influenced the direction of grant activities 
over this three- year period. These problems reflect issues and challenges in the field of severe social- 
emotional-behavioral disabilities and how they inpact on outreach assistance for model 
implementation rather than problems specific to this project. Conditions that gave rise to these 
problems and how the project responded are described below, by management objective. 

Problems in Dissemination, and Project Response 

The extremely large volume of requests for information about the model (and how to obtain 
assistance in implementing it) was not anticipated. Daily requests were received from individuals, 
program administrators, professionals, parents, or direct service providers nationally and 
internationally. The solutions were (a) an expansion and redesign of print materials into electronic 
media, (b) design of a web site with links to other resources, (c) sharing of materials and collaboration 
for translations in response to international requests for materials and guidance in model replication, 
and (d) extensive telecommunications to reduce actual staff travel for model dissemination purposes. 
To accomplish these new directions, it was necessary to redesign job descriptions for both key staff 
and for new personnel, primarily in use of advanced electronic communications and in computer 
graphics design. 

Problems in Planning for Model Implementation, and Project Response 
The project received many more requests for assistance in model inplementation than could be 
provided. This was particularly true for large programs, some with as many as 25-30 teams 
(approximately 75 individuals at one site) seeking extended in-depth, year long training. It would have 
been impossible to meet this need using the outreach model of intermittent, in-depth training and 
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follow-up at participating site visits for observations and debriefings with every individual teara The 
solution was to carefully plan for specific needs at each program with the site administrators. This 
planning included identifying a limited number of teams who volunteered for a one or two year pilot 
effort to assess the goodness of fit between the model and the established program. Only one of the 
21 sites was not able to continue implementation activities after initial training (this was due to 
turnover of both top administrators and teaching teams). 

Another problem in planning involved local concerns about how much additional paper work 
and time would be required of participants. Already over-loaded with paper work and record keeping, 
this question was raised at every site. A parallel concern was the question of "fit" between the model's 
assessment instrument for developing lEPs, Functional Behavioral Assessments, Behavioral 
Intervention Plans and the site's district requirements. These issues of balance between model 
requirements, limitations in project staff, overload of staff at local sites, and their expressed needs for 
inservice assistance were addressed during planning with administrators at each site. In the initial 
inservice training with participating teams these issues were frequently revisited. Most, but not all of 
the sites were able to blend model implementation requirements with local requirements. In each 
instance where it did not occur, local administrators and participating teams made the decision to 
include the models instruments for social-emotional-behavioral assessment as an add-on to local 
requirements for lEPs, FBAs and BIPs. 

Problems in Model Implementation, and Project Response 

Turnover of staff is an on-going problem throughout the field. We encountered numerous instances 
of absenteeism, staff resignations during the school year, and extended illnesses both physical and 
mental, causing shifts in job assignments and changes in teams participating in project activities for 
in-depth model implementation. Site administrators expressed their concern over this dilemma, which 
left them with new, inexperienced or untrained replacement staff throughout the year. This situation 
gave rise to a need for repeated introductory training sessions on the basics of model implementation, 
while other team members were ready for advanced skill development. 

To address this problem, a pilot initiative was designed to train local coordinators/supervisors 
at the implementation sites. The objectives were (a) to prepare these direct service leadership 
individuals to conduct introductory inservice training for their new staff and (b) to provide support 
to existing staff as they continued to acquire advanced skills for model implementation. 



Additional problems encountered during on-site activities for model implementation were (a) 
staff absentee rate (with difficulty finding substitutes); (b) supervisors styles of crisis management, 
often resulting in their intensive involvement directly with children in crisis rather than in guiding staff 
to more effective solutions; (c) fi*agile or non-existing coordination between families and school 
personnel (characteristic for many programs in this field); and (d) lack of substitutes/released time for 
participants during inservice visits by project instructors. This latter problem curtailed the extent to 
which instructors could provide debriefings and individual feedback to teams at some sites following 
observations. Frequently instructors would be asked to demonstrate a more effective strategy, to 
advise staff about a particularly difficult problem, or to meet with parents of a particular child. 
Sometimes, project instructors would be asked to actually cover a teacher’s group in a classroom 
while she/he was dealing with a serious crisis and had no other crisis backup. Instead of responding 
to these types of requests directly, project instructors provided coaching to the staff in solving these 
problems as a teara 

With limited on-site time for project instructors to address these implementation problems, 
a re-shaping of our technical assistance was undertaken during Project Year 2. Staff fi"om NEC*TAS 
helped us expand our long-distance communication links and instructional options to the 
implementation sites. The result was a web site, LIST-SERVE, teleconferencing, satellite hook-ups, 
and fi"equent phone consultation with site administrators and with participating teams. 

Problems in Interagency Collaboration, and Project Response 

This management objective received less proportional staff time than other activities yet the rapidly 
expanding need for close communication among agencies serving this population could have justified 
a full-time staff position. The special education/general education initiative for inclusion required 
considerable cross-agency work. Local and state level early childhood intervention programs also 
were facing the challenge of very young disruptive children with extremely difficult social-emotional- 
behavioral problems manifest in inclusive early childhood settings, child care, and foster care. The 
increasing incidence and severity of such problems served to further expand the need for interagency 
collaboration at local, regional, and state levels, including university and other state agency training 
programs. In addition, mental health agencies were beginning significant expansions into the schools 
with mental health services to troubled children and youth and their families. Payments for these 
services and coordination of treatment plans with educational plans became major interagency issues 



at several sites. 

Because the extent of this need was not anticipated in the original project proposal, it fell to 
core project staff to provide interagency collaboration in a limited way at state and regional levels, 
while our field-based instructors represented the project in coordination with local sites and agencies. 
They were asked frequently to attending meetings, participate in conferences, and make presentations. 
While these activities somewhat reduced the time project instructors spent in direct on-site assistance 
to participating teams, they were able to contribute to solutions for service delivery and personnel 
issues. 

Problems in Product Development, and Project Response 

The greatly increased number of requests for information and training for the Developmental 
Therapy-Developmental Teaching model, described above, required a considerable shift to electronic 
media to meet these needs. Without an increase in funding, product development and project 
resources for this management objective as originally proposed had to be redesigned. The original 
proposal anticipated production of three video tapes to introduce the model to audiences working 
with three age groups: early childhood, elementary school aged, and teens. We shifted resources to 
produce only two introductory videotapes, using the remaining resources to develop our first on-line 
course module to teach the basics for model implementation. 

Problems in the Pilot Program for Training Trainers, and Project Response 
Leadership individuals responsible for coordinating and supervising model implementation activities 
day-to-day expressed need for advanced skills and knowledge about the model. This was particularly 
evident to them between visits of the project instructor. Participating teams went to them for 
feedback, problem solving, support, and guidance as they worked to implement the model with 
children who had severe social-emotional-behavioral problems. With staff turnover, these leadership 
individuals also found themselves needing to repeat introductory level training for new personnel 
To address this need, during the second year of the project we focused on identifying leadership 
personnel at participating sites who wanted extended training to become certified trainers. Standards 
for acceptance into the program and five rigorous performance standards for certification were 
established. [See description in previous section on accomplishments.] Individual training programs 
were designed to meet these standards, and training was implemented during the latter half of Project 



Year 2. Leadership training included shared presentations with the project instructor at their own sites 
and at regional and state conferences. They also were required to observe and debrief with their own 
staff and the project instructor during site visits. Trainees expressed some concern that they would 
be unable to corr^lete all of the requirements for certification during the project funding period, but 
were assured that they could progress through their training at their own rate. At the time, it was 
unclear how long it would take these individuals to accorr^lish the certification requirements. In spite 
of this uncertainty, they continued. 

Problems in Project Evaluation, and Project Response 

This management objective presented the greatest problems for the project, as we originally 
anticipated conducting evaluation activities with the rigor of a research project. The evaluation design 
proposed in the original proposal was used, but problems inherent in field based data gathering 
presented obstacles requiring modifications in several of the proposed evaluation activities. Because 
the evaluation plan had both formative and summative aspects, evaluation was a significant time- 
intensive, on-going project activity. It became necessary to shift position responsibilities among 
project staff when the collection and maintenance of accurate field records became increasingly 
demanding and time consuming. 

The most difficult aspect of the design to fulfill was the assurance of reliability and validity 
of the observational performance data collected on participating teams and the children they served. 
At every site, project instructors reported the same types of difficulty in observing and rating the 
teams at work. For exarrple, when an instructor arrived at a site to observe a team's performance, it 
was not unusual to find (a) the group on a field trip, (b) a high number of children absent, (c) a key 
member of the team absent, (d) a substitute for the lead teacher (e) a non-representative activity such 
as lunch or study hall, (f) new staff, and/or (g) some children following a part-time schedule in an 
inclusive general education class, necessitating a split in the team as one staff person went along to 
assure that the inclusive experience was successful. 

Collection of reliable and valid data on the progress of children served during the project 
presented a different set of problems. One of the core requirements for model implementation is the 
accurate use of the DTORF-R rating procedures by the participating teams as they rate the social- 
emotional-behavioral development of every child in their group. The project staff did not do these 
ratings, but reviewed each team's completed ratings for accuracy. If discrepancies were evident, the 



instructor and the team met to revise the ratings. This procedure required the project instructor to 
have sufficient time when on site to observe each child in the program. The original evaluation plan 
specified collection of both baseline and intermittent DTORF-R measures on each child. Valid 
baselines were sometimes difficult to obtain from some teams because (a) they lacked a sufficient 
understanding of the instrument even though they had participated in the preliminary workshops (a 
core content requirement), (b) difficulties for project staff to obtain the basic denrographic 
information needed (such as file access) to describe the sample population, (c) carelessly con^leted 
ratings, (e) untrained staff participating in the rating, and/or (d) incomplete ratings. Collection of valid 
ratings repeated throughout the school year to document progress was also difficult as children (a) 
moved away, (b) were newly enrolled), (c) transferred to other programs, and/or (d) were absent 
during the rating periods. 

To assure reliability and validity of the data and confidence in the accuracy of the findings for 
participating teams and children, we had to accept smaller numbers in our samples. While this 
approach may have introduced bias into the sanple selection process, we chose to use samples which 
had reliable data, excluding those where data were incomplete or inaccurately collected. We believe 
the smaller samples are representative of the typical participants, children served, and sites. 

Another, less significant change in the original evaluation plan involved the discontinuation 
of administering the pre- post- knowledge test for participating teams. A decision was made at the 
end of the first year of the project that a participant knowledge test per se was not entirely suitable 
because of the intense focus of our technical assistance on actual, demonstrated skills and 
performance of participants. The paper - pencil test put some team members at a distinct disadvantage 
and resulted in low scores when their actual performance demonstrated understanding of the content. 
We believe that effective performance on the DTRITS requires understanding of the Developmental 
Therapy-Developmental Teaching knowledge base, and is a sufficient proxy for knowledge. Only in 
the instance when participants seek credit for the inservice did we use the knowledge test. 

IMPLICATIONS FOR POLICY, PRACTICES, AND RESEARCH 
There is deep, widespread concern among school personnel and parents about troubled children and 
teens. While national statistics reflect drops in juvenile crime rates, dramatic acts of violence by 
children against themselves and other children, children against teachers, and children against their 
own parents have raised awareness of society’s failure to met the social-emotional needs of its young 



people. The explosion of violent acts in schools during the past three years caused a surge in requests 
for information about the Developmental Therapy-Developmental Teaching model. We believe that 
the volume of requests we received reflects the deep concerns of those who work and live with 
seriously troubled children and youth on a daily basis. They are simply unprepared for the levels of 
complexity presented by this group of young people. Not only parents, but even experienced 
professionals are faced daily with hurdles requiring skills and understanding beyond the ordinary 
scope of parenting, mental health, and educational interventions. 

Recommendations for the Field 

The complexity of troubled young people demands an equally sophisticated, multidimensional 
approach with shared values and standards that transcend races and cultures. Providing for 
complexities involved in effective special education for this group of children and young people 
should be a central principle in policy and practice. Here are several recommendations that would 
follow from such a central principle: 

1. Program missions should be grounded in well established complementary theories 
about how children develop mentally healthy personalities, and include learning, 
valuing, relating, behaving with responsible self-control, and basic thinking and 
problem-solving. 

2. Programs should be conducted with seamless components for mental health 
interventions, and include involvement with other major social institutions that shape 
children’s lives - families, childcare, law, government, recreation, and spiritual life. 

3. Assessments should be based on procedures shown to be reliable and valid for 
identifying child's current assets in each of the areas addressed in the scope of the 
intervention program. 

4. In planning a child’s intervention program, defined procedures should be used for 
gathering and analyzing past experiences to more fully understand the inpact on a 
child’s current status. 
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5. Advanced skill training with demonstrated proficiencies in developmentally and 
emotionally appropriate practices, human relationships, and sustained practice of 
mission standards should be required for anyone working in this field — including 
professionals and paraprofessionals in special education, mental health, and general 
education. 

6. Criteria for child progress should be established with defined outcomes having 
practical and theoretical validity. 

7. On-going inquiry into the presumed effectiveness of every practice with every student 
should be part of every program. 

The field should be held to such basic standards for children and youth with severe social-emotional- 
behavioral disabilities. 

Recommendations for Effective Outreach and Technical Assistance 

Project experiences, problems encountered, and feedback from front-line practitioners over the past 
three years suggest numerous ways to assist individuals and programs at the local level in meeting 
the needs of this difficult-to-serve group of young people. 

1. Family involvement in intervention programs . Family involvement was initially low at the 
participating sites, reflecting similar widespread problems in the field. We found that by putting a 
priority on this, we were able to make some change in attitudes and practices. Specifically, we found 
that parents, encouraged by the local staff and our project instructors to participate in the basic skills 
workshops along with program staff, were responsive and could utilize the training at home. We also 
found that this co-participation built greater understanding between staff and families. Feedback from 
some project participants noted that parents who participated with their child’s team in rating social- 
emotional-behavioral development were increasingly positive about their child’s abilities and potential 
for progress in the program. 
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There is concrete shift in attitude based on how we are inpacting our 
parents in that process. When we do DTORF assessnnents with 
parents we hear comments like "I thought my child was really 
screwed up (and that was the nicest comment). I found that my child 
was really quite normal but that my expectations were to high." 



While such highly positive changes were not evident at all sites, we believe that by putting greater 
ertphasis on parents as team members, outreach projects can contribute significantly to enhancing 
constructive family involvement for a child’s benefit. 

2 Skilled local leadership. The greatest benefits from model inplementation accrue when local 
programs have their own supervisors/coordinators trained to high levels of proficiency and knowledge 
about the model. At sites where we had a leadership trainee actively participating in model 
inplementation as a trainer-in-training, there was greater progress by the teams in demonstrating and 
sustaining effective model practices. There also appeared to be more confidence among the direct 
service teams to attenpt new or inproved practices when the coordinator was actively involved both 
during the instructor’s site visit and during the interim between visits. Finally, assessment of child 
progress was more accurately completed at sites where there was an active leadership person in 
training. 

3. On-going site evaluations. Each site should work to accumulate a database to build their 
own normative expectations about child progress in that program. However, teams in intervention 
programs are not typically enamored ofdata collection processes, justifiably, as additionalpaperwork 
and accuracy are necessary. However on-going evaluation of each child’s progress is essential if 
program quality is to be maintained. We observed that teams were able to modify their day-to-day 
practices with more precision at sites where more frequent child progress data were collected. They 
adjusted their practices as children made progress. In contrast, at sites where assessments were made 
only at the beginning and end of the year, program practices were not as readily changed as children 
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changed. When this occurs, children tend to plateau and a program could inadvertently contribute to 
a ceiling on greater social-emotional-behavioral development. We found that considerable outreach 
effort needs to be put into helping local administrators and coordinators put basic evaluation 
procedures in place. When local staff and parents (a) see evaluation results in formats easy-to- 
understand and interpret at a glance, (b) receive supportive assistance in using the results in practical 
ways to improve classroom conditions, and (c) are assured that their own value is not threatened by 
the results, there appears to be greater commitment to being a part of ongoing program evaluation. 



Recommendations for Outreach 

The degree of flexibility in current OSEP guidelines for conducting discretionary grant-funded 
projects is reasonable and helpful to outreach activities. With a field changing as rapidly as it did 
during the three years of this project, flexibility in modifying staffing patterns, staff assignments, tasks 
to be accomplished, and procedures was essential for successful accomplishment of the project goals. 
We found the meetings in Washington extremely helpful in keeping up with current trends and new 
innovations — especially those that focused on our area of severe social-emotional-behavioral 
disabilities and technical problems of documenting intervention effects. We also found that contact 
with the larger national technical assistance projects were of help, especially those that came from 
allied fields involved with mental health or technological issues. Greater contact with grant officers 
by allowing them to visit funded projects and implementation sites could also promote greater 
utilization of proven models and practices nationally. 

Because the area of early childhood special education is distinctly different from general 
special education, especially in the area of severe emotional-behavioral disabilities and Autism 
Spectrum Disorders, we found it necessary to apply for a second outreach project to obtain additional 
resources for more targeted outreach to programs serving these very difficult-to-manage young 
children. That second outreach funding contributed immensely to the overall success of our outreach 
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effort by allowing us to provide a coordinated flow of assistance across age ranges, especially for 
programs that serve both early childhood and school aged students. Such multiple-grant funding 
assists outreach programs in maintaining a broad scope of expertness necessary to respond effectively 
to needs in the field. 

At present funding levels, only core outreach services with intermittent assistance over a 
three- year period can be provided. Yet, requests for assistance far exceed capacity to respond both 
within specific sites where irr^lementation is occurring and at new sites where entire school districts 
or n»ntal health programs seek assistance. Expansion of federal funding for outreach, both in dollar 
amounts and in funding periods (preferably firom three to five years) would enable outreach programs 
to expand, sustain on-going efforts, and increase the depth of skills at participating sites. Increased 
funding levels would also allow for increased FTE for greater project involvement in interagency 
planning at state, local, and regional levels. With educational reforms at high levels, it seems essential 
that model outreach programs contribute to planning for educational in^rovements. 

Finally, our single greatest number of requests for on-site outreach assistance came firom 
direct service providers who wanted to see model practices in action. They expressed a need to 
observe model practices demonstrated effective with children who had challenging behaviors similar 
to the ones they experience daily. Whenever possible, project instructors identified staff in local 
programs that were demonstrating proficiency and success with model practices. Yet, other teams 
at the same program seldom had released time to observe these practices and learn firom them. We 
also made arrangements for a few direct service providers to visit other programs to see model 
practices. But these opportunities were few because direct service personnel can seldom be released. 

Project instructors were also asked firequently to demonstrate specific strategies when making 
site visits. We did not encourage this because of project focus on staff training and not direct service 
delivery. However, we endorse the idea that skill acquisition is easier when there are opportunities 
to observe and model effective practices. A multiplier effect is created when local demonstration sites 
are available for observing and modeling effective practices. We believe that observations of 
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successful practices by direct service peers is among the most effective and cost efficient activities 
for a model outreach program. However, current OSEP ceilings on funding levels for model outreach 
projects make it difficult, if not impossible, to mount a project component to provide these 
opportunities. We recommend several policy standards for OSEP to consider: 

/ Expand maximum amount available for project proposals to include a specific 
component for direct service demonstration activities. 

/ Encourage outreach projects to offer extended regional summer institutes where 
direct service providers could participate in-depth as observers and team members in 
a successful demonstration program. 

/ Allow outreach projects to offer stipends and expenses for participating at in-depth 
training institutes for inplementing nwdel practices. 

/ Allow funds for partial payments to demonstration teachers at model sites and related 

program expenses such as transportation of children to the demonstration site. 

/ Encourage project FTE for project instructors to coordinate direct service programs 
for children and supervise learning experiences for direct service trainees at summer 
demonstration components of the outreach project. 

/ Allow funds, including travel and substitutes, for direct service providers to visit 
demonstration sites for short visits. 




In summary, this project has shown that extended on-site, in-depth, extended outreach 
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assistance will result in improved program quality and skill acquisition by direct service providers, 
supervisors, coordinators, children, and their families. The lesson to be learned is that even more can 
be gained from these expenditures in the future if closer links are made available between an outreach 
project, local implementation programs, and high quality demonstration programs. 



Tables 1 - 11 
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Table 1. Observational Performance Ratings of 28 Direct Service Teams 
(79 Participants at 9 Sites After Initial Model Implementation) 



Participant 

Team 


DTRITS Scores and 
Proficiency Levels Achieved 


Team 

ID 


n 

individuals 


Highly 

Effective 

90-100% 


Effective 

70-89% 


Adequate 

50-69% 


Below 

Passing 

<50% 


031 


2 




89 






032 


2 




85 






033 


3 




100 






041 


2 








38 


042 


3 






56 




043 ■ 


3 






56 




051 


2 








19 


052 


3 


100 








801 


2 






61 




802 


5 




79 






803 


2 






68 




804 


2 








43 


901 


2 




80 






902 


2 








18 


903 


2 




72 






904 


2 








26 


1001 


2 






68 




1002 


3 




80 






1003 


3 








30 


1101 


3 




88 






1102 


3 


91 








1103 


5 




82 






1104 


5 




71 






1105 


3 




79 






1107 


5 




89 






1108 


3 




86 






1701 


2 






67 




2201 


3 




79 






TOTAL 
28 teams, 

79 individuals 


2 teams, 

6 individuals 


14 teams, 

46 individuals 


6 teams, 

14 individuals 


6 teams, 

13 individuals 
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Table 2. Regional Associates in Training: Progress toward Certification (N = 28) 
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Table 3. Pre- Post* Knowledge Test Scores for 11 Leadership Trainees 



Trainee 


Pre-test 


Post-test 


#2 


66 


73 


#6 


73 


83* 


#7 


71 


86* 


#11 


66 


83* 


#12 


81* 


85* 


#13 


65 


83* 


#15 


62 


81* 


#16 


61 


77 


#17 


80* 


NA 


#22 


66 


79 


#27 


67 


73 



♦Performance criterion = >79 
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Table 4. Workshop Evaluation Summaries for Project Years 1 & 2 (N -172 respondents at 8 
workshops) 

E valuation Statement Average Rating 

(scale = 5 to 1) 



1. The material presented has been ... 

Very Benrficial (5) to No Benefit (1) 



4.73 



2. The workshop was ... 

Well Organized (5) to Disorganized (1) 



4.71 



3. My general impression of the workshop is ... 

that it was 

Excellent (5) to Poor (1) 

4. This workshop met my need ... 

Very Well (5) to Not at All (1) 



Table 5. Workshop Evaluation Summaries for Project Year 3 (N - 643 respondents at 23 
workshops) 

E valuation Statement Average Rating 

(scale = 5 to 1) 



1. The material presented has been ... 

Very Bentficial (5) to No Bentfit (1) 



2. The workshop was ... 

Well Organized (5) to Disorganized (1) 

3. My general impression of the workshop is ... 

that it was 

Excellent (5) to Poor (1) 



4. This workshop met my need ... 

Very Well (5) to Not at All (1) 
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Table 6. Follow-up Evaluation by Direct Service Teams at Model Implementation Sites 

(N = 23) 



Training Activity 


n 

responses 


Average Rating Range 

(5-1 scale)* 


Group Training Sessions 


19 


4.26 


3-5 


Instructor's Observations in 
Your Class 


16 


4.00 


3-5 


Feedback! Debriefings 


16 


3.87 


3-5 


Written Feedback 


6 


3.67 


'3-4 


Other 

"Training at demonstration 
site"; "Staff development"; 
"Reading text" 


3 


4.50 


4-5 



*5 = Very Helpful to 1 = Not Helpful 
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Table 7. Characteristics of Children with Repeated Performance Measures at 9 Representative 
Sites Where In-depth Training for Model Implementation Occurred. 

(N = 199) 



Site 


C 


D 


E 


G 


H 


I 


J 


M 


N 


Total 


n 


8 


39 


7 


35 


37 


21 


31 


11 


10 


199 


Average Baseline Age 


(months) 


103 


145 


100 


147 


153 


147 


66 


113 


101 


— 


(years) 


9 


12 


8 


12 


13 


12 


5.5 


9 


8 


— 


Age Range (yrs.) 


7-10 


7-16 


6-11 


5-20 


0-18 


7-18 


3-9 


6-11 


5-12 




Gender 


Boys 


7 


37 


7 


28 


29 


18 


27 


8 


9 


170 


Girls 


1 


2 


0 


7 


8 


3 


4 


3 


1 


29 


Race 


White 


4 


21 


6 


33 


33 


17 


18 


4 


10 


146 


Black 


4 


18 


1 


0 


1 


2 


10 


6 


0 


42 


Asian 


0 


0 


0 


0 


1 


0 


1 


1 


0 


3 


Hispanic 


0 


0 


0 


0 


0 


2 


1 


0 


0 


3 


Mixed 


0 


0 


0 


2 


2 


0 


1 


0 


0 


5 


Disability 

(primary) 


Emotional/behavioral 


2 


35 


2 


34 


30 


21 


5 


6 


0 


135 


Developmental 


0 


1 


0 


0 


1 


0 


2 


0 


1 


5 


Intellectual 


0 


1 


0 


1 


3 


0 


0 


0 


3 


8 


Speech/language 


3 


0 


0 


0 


0 


0 


0 


0 


1 


4 


Autism 


3 


1 


5 


0 


2 


0 


23 


5 


4 


43 


Other 


0 


1 


0 


0 


0 


0 


1 


0 


0 


2 


Secondary Disability 


Yes 


6 


24 


0 


9 


18 


0 


3 


3 


5 


68 


No 


2 


15 


7 


26 


19 


21 


28 


8 


5 


131 


Severity at Baseline 


None 


0 


1 


0 


1 


2 


2 


0 


1 


1 


8 


Mild 


0 


13 


2 


13 


14 


12 


3 


2 


0 


63 


Moderate 


4 


24 


5 


20 


19 


7 


17 


5 


8 


106 


Severe 


4 


0 


0 


0 


1 


0 


11 


3 


1 


19 


Missing data 


0 


1 


0 


1 


1 


0 


0 


0 


0 


3 


Severity at Time 2 


None 


0 


4 


0 


2 


2 


3 


2 


1 


1 


15 


Mild 


0 


12 


4 


19 


19 


11 


5 


3 


2 


75 


Moderate 


4 


23 


3 


14 


15 


7 


12 


5 


7 


90 


Severe 


4 


0 


0 


0 


1 


0 


12 


2 


0 


19 


Missing data 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


Months: 


Base to Time 2 


4.4 


5.4 


6.7 


5.5 


5.9 


6.2 


8.1 


5.4 


3.6 


— 
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Table 8. Students' Performance Scores at Baseline and at Time 2 During Staff Training for Model Replication at 9 Sites 
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Table 10. Comparison of Children's Actual DTORF-R Scores and Extrapolated Scores Assuming No Intervention. 
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Table 11. Level of Administrative Support for Replication at 9 Representative Sites 
Participating in Project Evaluation 



Site 


Score* 


Replication Level 


C 


30 


Exertplary Model Demonstration 


D 


17 


Model Adoption 


E 


25 


Model Adoption 


G 


13 


Basic Elements Used 


H 


30 


Exertplary Model Demonstration 


I 


12 


Basic Elements Used 


J 


19 


Model Adoption 


M 


13 


Basic Elements Used 


N 


21 


Model Adoption 



*Score indicates number of administrative elements in place for model replication 
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APPENDIX A 

Administrative Organization Unit 
Within the University 



Appendix A. 

Organization of the 

COLLEGE OF FAMILY AND CONSUMER SCIENCES 
Academic Programs, Research, and Service 
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Fee Schedules 
Used with Local Sites 
for 

Cost Sharing 
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Appendix B. 

Fee Schedules Used with Local Sites for Cost Sharing 



Developmental Therapy-Teaching Programs 



P.O. Box 5153 

Athens, Georgia 30604-5153 



SCHEDULE OF FEES 



Phone; 706-369-5689 
Fax: 706-369-5690 
e-mail; mmwood@arches. uga.edu 



Workshops, Site Development Assistance, or In-Service Training 



Fees and Expenses Paid by Local Proeram 



Workshop by Topic 

Limited to 1 or 2 full days 



Institute provides workshop materials. 



For 30 participants or less: $150.00 (one-time 
fee), plus 1 instructor’s fee per day & travel 
expenses.* 

For more than 30 participants: $200 (one-time 
fee) plus 2 instructors’ fees per day & their 
travel expenses.* 



Site Development Assistance Travel expenses for 1 consultant. (Number of 

Overview presentation, on-site needs days to be negotiated.) 

assessment, program observations, 
consultation, and planning for model 
utilization. 



Institute provides consultant & materials. 

Extended Inservice for Implementation 

Year-long assistance, including 3 or 4 full 
day in-depth workshops and on-site individual 
team follow-up visits. 



Site provides copies of curriculum materials 
and released time for teams participating. 

Institute provides two certified instructors 
& workshop materials. 



One-time fee negotiated according to number 
of teams participating: 

• 4 teams or less = $300 per site, 

• over 4 teams = $600 per site, 

plus travel expenses for 2 instructors. 
Number of days to be negotiated. 

About $100 per team for materials, including 
DTORF-R assessment kit & curriculum guide. 



*A11 consultants and instructors have successfully completed the National Developmental Therapy 
Leadership Training Program and are certified as both Developmental Therapy-Teaching 
demonstration teachers and as staff instructors. The per instructor fee per day (plus travel expenses) 
paid by a local program for a topical workshop is $500 when the instructor has the doctorate and 
$250.00 without the doctorate. For other forms of technical assistance, consultation, and inservice 
training the Institute provides instructors’ fees. 

Travel expenses include u-avel, accommodations, and up to $35.00 per day for meals. 
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APPENDIX C 
Content Map 
for 

Developmental Therapy- 
Developmental Teaching 
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Appendix C 
CONTENT MAP 

For Developmental Therapy-Teaching 

Instructional Modules for Inservice Training 



PART I 



Introduction to This Approach 



A Module 1 ' 

Graphic: "Six Frequently Asked Questions About This Approach" 

Ql: "What is the program focus?" 

Q2: "Why are change and growth built into the program?" 

Q3: "How does this approach promote success-producmg behavior?" 
Q4: "How does it motivate students to become involved?" 

Q5; "What is required to use this approach?" 

Q6; "How do you know the program is effective?" 



Ql. Program Focus 

Q: What fundamental beliefs about troubled children guide the program? 

Graphic: = "Behave, Speak, Feel, Relate, and Think" 

Graphic: = "Four Foundation Beliefs and Program Implications" 

{Or group cards for matching beliefs & implications} 

Q2. Instructional Goals for Change and Growth 

Q; "How are change and growth built into the program?" 

Graphic: = "The Broad Sequence of Instructional Goals, Stages One - Five" 

(Figure 1.3 on page 8) 

Q: "What curriculum content is included to achieve these goals?" 

Graphic: = Doing, Saying, Caring, and Thinking" (Figure 2.2 page 34) 

Q3. Programmatic Changes for Success-Producing Behavior 

Q: "How do changes m adult roles and intervention strategies promote social- 
emotional-behavioral successes?" 

Q: "How do changes in leai'iiing envii-onments and experiences promote social- 
emotional-behavioral successes?" 

Graphic: = "Summary of Program Stages" (Figure 1.4 on page 10) 

Video: "Introduction to Developmental Therapy-Teaching - Little Kids to 
Teens" (use either preschool or school age version) 

Or . use "Roles of Adults" video to illustrate adults behavior and program activities. 
Stages One to Five 



‘a = Modules typically included as basic content for initial and middle phases of acquiring skills for using 
Developmental Therapy-Teaching. Selection of modues and order of use in inservice is determined by the needs of 
participants at the beginning of uaining and may be modified as uaining progresses. 
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Q4. Motivating a Student to Become Involved 

Q: "How do self-esteem, identify, and personal responsibility fit into this approach?" 
Graphic: = "Understanding a Student's Heart and Head" (List on page xii) 

Q5. Program Implementation 

Q: "Why combine Therapy' and 'Teaching'?" 

Q: "Where can this approach be used?" 

Q: "Which children benefit?" 

Q: "Who can learn this approach?" 

Q: "What special equipment and materials are needed?" 

Q: "How are parents involved?" 

Q: "How are cultural, age and family values addressed?" 

Q: "What place does academic instruction have?" 

Q: "How can other cumculum be included?" 

Graphic: = "More Questions" 

Q6. Documenting Program Effectiveness 

Q: "What place does evaluation have in the program?" 

Q: "Can social-emotional-behavioral growth be documented?" 

Q: "Can student gains be attributed to program intervention?" 

Graphic: = "The Criterion - Referenced Evaluation System" (Figure 3.5, p. 72) 
Graphic: = "The Example of a Field-Based Research Design" 

6 Graphics: = "Students A - F" 



PART 1 

Corresponding Readings: N ote: 

Preface, xii - xiii 
Chapter 1, pages 7-16 

Figures 1.3, 1.4, 1.5, 1.8 
Chapter 3, pages 55 - 61, 70 - 78, 
Figure 3.5 



Note: All reading references are for Developmental tlierapy-Developmerital Teaching (1996) unless otherwise indicated. 
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PART 2 

Using the DTORF-R 



A Module 2 

• Uses of the Instrument 

Q; "What is the DTORF-R?" 

Graphic: = (pages 1 - 2*) 

• Rating Procedure 

Q: "What is the procedure for using it with reliability?" 

Slide/audio, "Instructions 

"Review the Basics" (pages 18 - 25, 30*) 

Practice Cases (pages 26 - 29, 31 - 36*) 

Module 3 

• More Practice Cases 

Q: "Rating from a written description: Charlie" (pages 38 - 39*) 

"Frank" (pages 42 - 43*), "Donna" (pages 46 - 47*) 

Module 4 

• Content analysis of the Instrument Subscales 

Q: "What content scope and item sequences are included?" (Pages 2 - 8, 25**) 

Module 5 

• Documenting Student Progress, Part 1 

Q: "Is the student making desired gains, specified on the lEP?" 

Graphic: = (pages 11 -17*) 

• Documenting Progress, Part 2 

Q: "Can gains be attributed to the intervention?" 

Graphic = (pages 10 - 13**) 



PART 2 

Corresponding Readings: 

Chapter 3, pages, 61, 66 - 70 

* In Users Manual for the DTORF-R: 

**In Technical Manual for the DTORF-R: 
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PARTS 3 - 5 
Introduce each part 
using the transparency of the 
Developmental Therapy-Teaching Logo 

PART 3 

Healthy Social-Emotional Development for Typically Developing Age Peers 
A Module 6 

• Central Concerns and Values of Each Age Group 

Q: "What's really important to me?" 

Q: "What do I need from adults?" 

Graphic: = "A Child's Expanding Spirit" 

Q: "What do children this age typically value as 'satisfactory'? 

Graphic: = "The Sequence of Values" pages 41 - 43 (Figure 2.3 on p. 42) 

• Key Social-Emotional Processes for Each Age Group 

Q: "What typical social-emotional processes occur in each age group?" 

5 Graphics: = "Stage Charts" (pp. 180, 204, 233, 264, 294) 

[Also 5 laminated charts for group use] 

A Module 7 

• Developmental Anxieties of Each Age Group 

Q; "What central developmental anxiety is experienced by typically developing peers 
in each age group?" 

Graphic: = "How Typical Developmental Anxieties Emerge" (p. 47) 



PART 3 

Corresponding Readings: 

Chapter 2, pages 32 - 38, 41 - 48 
Chapter 3 

Appendix 4, pages 349 - 354 
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PART 4 

Decoding Behavior 



A Module 8 

• Review Underlying Social-Emotional Needs for a Student's Age Peers 
Q: "What is this student's age?" 

Q: "What are typically developing age peers experiencing?" 

Graphic: "Developmental Anxieties" (list on page 47) 

Or review "How Typical Developmental Anxieties Emerge" (page 47) 

Q: "What concerns, anxieties, and approach to problems are typical at each stage for 
social-emotional development?" 

5 Graphics: = "Student in Brief for age and stage (pp. 177, 201, 232, 259, 289) 



Module 9 

• Identify Other Special Developmental and Emotional Needs of Each Student 

Q: What special factors may be producing developmental anxieties and emotional 
needs in an individual? (Refer to history, clinical assessments, current interests, habits, 
and behavior) 

Practice Exercise: "Roger" (pages 62 - 66) 

Module 10 

• Identify Defense Mechanisms ui Observed Behavior 

Q: "Which defenses are being used by this student consistently?" 

Q: "Which anxieties are being protected?" 

Q: "How intensely are the defenses used to obtain emotional protection?" 

Graphic: = "The Process of Adjustment" 

Graphic: - " Defense Mechanisms" (pages 48 - 49) 

Practice Exercise: "Identify the Defense Mechanisms" (pages 50-51) 

Module 11 

• The "Existential Crisis" 

Q: "How important is the 'existential crisis'?" 

Q: "Are typically developing age peers going through the existential crisis?" 

Q: "Has this student passed through it?" 

3 Graphics: ="A Preexistential Student Views Adults" (page 39) 

"A Student in the Existential Crisis Views Adults" (page 40) 

"A Postexistential Student Views Adults" (page 41) 



PART 4 

Conespondintt Readings: 
Chapter 2, pages 38-51 
Chapter 3, pages 62 - 66 
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PARTS 

Materials and Activities 



A Module 12 

• Emotionally Appropriate Materials and Activities 

Q: "What content wUl motivate each individual?" 

Graphic: = "How the Emotional Memory Bank Works (Figure 4.5, page 91) 

Q: "How is emotional content selected to motivate and alleviate anxieties and 
concerns? 

4 Graphics: = "Content Themes" (List on page 92) 



A Module 13 

• Developmentallv Appropriate Materials and Activities 

Q: "What schedules, activities and materials will promote mastery of the selected 
Developmental Teaching Objectives? 

Graphic: = "Planning for Specific Objectives (Figure 4.2 on page 86) 

Graphic: = "How Materials Change with Development" (Figure 4.4 on p. 90) 



A Module 14 

• Putting Social-Emotional Content into a General Curriculum 

Q: "What activities and materials can promote mastery of Developmental Teaching 
Objectives within an existing program? 

Graphic: = "Examples of Content in Typical and Special Classes (Figure 4.3, 
page 88) 

Or 5 Graphics: = (Lists on pages 195, 216 - 217, 251, 275, 303) 

Module 15 

• Examples in the Language Arts 

Graphic: = Developmental Sequences in Language Arts (List on page 98) 

• Examples in Children's Literature 

Graphic: = "Criteria for Selecting a Storybook" (List from notes) 

• Examples of Teacher-Made Story Books 

Graphic: = "Steps in Designing a Teaching-Made Storybook" (List from notes) 

• Examples of Teens 

Graphic: = "Content themes for Teens" (Figure 12.2 on page 305) 



PART 5 

Corresponding Readings: 

Chapter 4, pages 81 - 94, 98 - 100 
Chapter 8, page 195 
Chapter 9, pages 216 - 217 
Chapter 10, page 251 
Chapter 1 1, page 275 
Chapter 12, pp. 303, 305 309 
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PART 6 

Positive Behavior Management 



A Module 16 

• Effective Discipline and Behavior Management 

Q: "What basic guidelines apply for children and teens of all ages?" 
Graphic: = "Four Keys to Successful Behavior Management" (p. 128) 



A Module 17 

• Positive Rather Than Negative Behavioral Results 

Graphic: = "6 Steps in Designing a Positive Behavior Management Plan" (page 
111 ) 



A Module 18 

• Positive Behavior Management Strategies Matched to Stage of Development 
Q; "What positive management strategies will be most effective? 

2 Graphics: = "Most Frequently Used Management Strategies" (Figure 6.1, 
p.l29), "Less Frequently Used Management Strategies" (Figure 6.2, p. 130) 
[Also, group activity cards matching strategies with definitions and stages] 



Module 19 

• Students' Changing view of Authority and Responsibility 

Q: "How do students change fi'om external control to personal responsible for 
behavior? 

Graphic: = "Who is Responsible?" (New) 

Q: "What adult behavior is needed to assist students take increasing personal 
responsibility?" 

Graphic: = "Elements in Building a Relationship" (Figure 7.1 on page 159) 



Module 20 

• Group Dynamics 

Q: "What forms of social power are used by students and adults alike?" 

Graphic: = "Social Power" (list & definitions on page 159) 

Q: "Which group role is held by each individual in the group?" 

Graphic: = "Roles of Individuals in Groups" (Figure on page 1 13) 
and "Social Power in Groups" (Figure 10.3 on page 248) 

Q: "What changes are needed in social power and group roles to foster positive 
behavior of group members?" 

2 Graphics: = " Chart of Behavioral Relationships Among Six Students" (Figure 
10.4 on page 249 and "What is Evident?" (Accompanying questions) 



PART 6 

Corresponding Readings: 

Stage Chapters, pp. 107 - 122, 125 - 149, 
158 - 163, 246 - 250 
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APPENDIX D 

Strategic Plan for 
Extending Outreach 
Via Distance Learning 
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Appendix D. 

Strategic Plan for Extending Outreach 
Via Distance Learning 

I. System Variables 

Target Audiences 

What groups do we want to reach? 

What are their needs? 

What are the conditions and constraints around their learning? 

Content Domains, Goals, and Modules 
What "awareness* information is needed? 

What basic content skills and knowledge are needed? 

What advanced content skills and knowledge are needed? 

Delivery Options 

What delivery options are currently available? 

What instructional characteristics and benefits does each offer? 

Which options are suitable for our target audiences? 

Which options fit our content domains, goals, and modules? 

II. Design Variables 

Accessibility and Program Capability 

How accessible are delivery options for our program? 

How accessible are delivery options for our target audiences? 

What levels of staff skills in technology are needed for each option? 

Relative Costs and BenePits Among Alternatives 
What are the unit cost estimates for delivery options? 

Which options offer feasible alternatives? 

The Strategic Plan 
What are the priorities? 

What are the steps? 

What additional resources will we need (costs, personnel, TA) ? 

What timelines are realistic for implementation ? 

m. Implementation Variables 

Design of Instructional Strategies for Selected Options 

What are the unique learning characteristics utilized by each option? 

How much interaction will be included? 

What technological materials will be used? 

What support resources will be provided? 

Who will facilitate the learning? 

Who will handle the technology? 

What instructional feedback will be provided to the learner? 

What instructional evaluation and follow-up will be provided? 

Design of Outcome Measures 

What are the expectations and outcomes wanted by participants? 

What are the expectations and outcomes wanted by our program? 

What measures will be used to evaluate the amount of these expectations? 

How effective and useful are the implemented options in accomplishing these expectations? 
What impact does the outcome have on services to children? 
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Appendix E. 

Developmental Therapy ■ Teaching Training Trainers Program: Regional Associates 

October 1 , 1 996 - September 30, 1 999 



Name 


Position 


Location 


Paul Baker 


Coordinator 


Mountainbrook Psychoeducational Center, 
Canton GA 


Larry 8eye 


Director 


Behavioral Health Resources, 
Olympia WA 


Judith Bondurant-Utz* 


Professor, 

Exceptional Education Department 


Buffalo State College, 
Buffalo NY 


Jane Butier-Ni>c* 


Therapeutic Preschool Educator 


Adams Elementary School, 
Yakima WA 


Charieen Cain 


ED/BD Teacher 


Maine School Administrative District #40, 
Waldoboro ME 


Patrida Copeland* 


Therapeutic Child Development 
Program Supervisor 


Learning Tree, 
Bremerton WA 


Cynthia Edwards* 


Resource Consultant 
Early Intervention Center 


Positive Education Program (PEP), 
Cleveland OH 


Muazzez Ercn 


Director, Family Development Center 


Healthy Families of Clallam County, 
Port Angeles WA 


Pamela Fox* 


Head Start Training and Resource 
Specialist 


Audubon Area Community Services, Inc., 
Owensboro KY 


Andrea Cullen 


Consultant 


Columbus GA 


Barry Oinnis 


Program Director. Group Homes 


Positive Education Program, 
Cleveland OH 


Amy Hepburn 


Family Advocate 


Shoreline School District, Seattle WA 




LouAnn Hepp 


Behavior Specialist 


Issaquah School District, I 
ssaquah WA 


Theresa Johnston 


Program Development Coordinator 


Center for Special Needs, PEP, 
Cleveland OH 


Kelley Simmons Jones* 


Executive Director and Therapist 


Monarch Therapeutic Learning Center, 
Lacey WA 


Scotty Jones* 


Program Supervisor and Staff Trainer 


Monarch Therapeutic Learning Center, 
Lacey WA 


Dan Kettwig 


Behavior Specialist 


Port Orchard WA 


Dennis Koenig 


School Psychologist 


West Shore Day Treatment Center, PEP, 
Cleveland OH 


Ru Kirk 


Clinical Social Worker 


Port Townsend WA 
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Name 


Position 


Location 


Pamela Massingale 


Coordinator 


Mounatainbrook Psychoeducational Center, 
Dalton GA 


Linda Middleton* 


Executive Director 


Sunshine and Rainbows Child Development Center, 
Forks WA 


Billie Navojosky* 


Program Coordinator 
Early Intervention Center 


Positive Education Program (PEP), 
Cleveland OH 


Patty Orona 


foster Parent Trainer 


Port Angeles WA 


Mary Perkins* 


Early Childhood Coordinator 


Educational Service District 113, 
Olympia WA 


Susie Sarachman 


Education Plus Counselor 


ATTN Special Services / McKinney Elementary School 
and Behavioral Health Resources, 

Olympia WA 


Claudia Valore 


Program Coordinator 


Hopewell Day Treatment Center, PEP, 
Cleveland OH 


Suzan Wambold 


family Development Specialist 


The Casey family Program, 
Tacoma WA 


Nancy Wheeler 


Speech/Language Pathologist 


Clover Park Schools, 
Lakewood WA 



* Training shared with our Early Childhood Outreach project 
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Appendix F 

DEVELOPMENTAL THERAPY -TEACHING PROGRAMS 

TRAINING OF TRAINERS 

Regional Associate Certiflcation 
Competency Evaluation Standards 



Competencies 


Evaluation Procedures 


Performance Standards 


Knowledge of developmental 


Developmental Therapy Knowledge 
Test (50 items) 


88% (44 items) correct, or greater 


theory, research, and resulting 
curriculum practices 


Reliable use of a rating procedure 
to identify social-emotional- 
behavioral objectives for children's 
lEPs 


Developmental Teaching 
Objectives Rating Form-Revised 
(DTORF-R; 171 items) scored 
against a rating by a nationally 
certified instructor 


90% agreement, item-by-item, 
against the instructor’s rating 


Reliable use of an observational 
rating form for assessing teachers’ 
classroom competencies 


Developmental Therapy Rating 
Inventory of Teacher Skills 
(DTRITS; 212 perfornnance items) 
scored against a nationally certified 
instructor scoring protocol during a 
paired observation 


80% agreement, item-by-item, 
against the instructor’s rating 


Field supervision of a team 
beginning to use Developmental 
Therapy-Teaching 


Evaluation of Trainer’ s Field 
Skills completed by participating 
teacher (8 items) 


Average rating of 4 or better on a 
5 -point scale 


Group instruction of staff 
participating in a Developmental 
Therapy-Teaching workshop 


• Workshop Evaluation Form 
(4 items) 

• Evaluation of Session Leader 
rated by a nationally certified 
instructor in Developmental 
Therapy-Teaching 

(18 items) 


Average of 4 or better on a 5- 
point scale 
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L* what observations do you have about your effect (directly or indirectly) on the children, families, 
\ff, and programs you work with? 
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APPENDIX I 

Open-Ended Responses 
of 

Direct Service Participants 
to 

Evaluation Survey 



Appendix I. 

EVALUATION OF TRAINING: SUMMARY BY DIRECT SERVICE PARTICIPANTS 

(N=23) 



What can you do now that you could not do before? CN=21) 

A. Understand the developmental stages of children/students 

• Chart the stage of development of each student 

• Tell what developmental stage students are in (5) 

B. Assess social-emotional-behavioral competence 

• Relate different age levels to the DTORF stages 

• Now know how the DTORF works and how it will be used in our program 

• Give the DTORF correctly (2) 

• Fill out a DTORF without getting sweaty palms 

C. Program more effectively 

• Think about children in term of readiness for activities and skills. 

• Plan activities and groups according to developmental levels (3) that move kids through 
developmental levels £r skills 

• Better integrate therapeutic interventions into academic lessons 

• Develop plans that work with stages (2) 

• Assume a wider “scope' of delivery to meet very specific objectives related to 
developmental milestones 

D. Use management strategies effectively 

• Deal with behavior problems better 

• [Reading the book has given me the ability to] utilize which behavior modification and 
management programs need to be used in a stage III classroom 

• Find the underlying conflict in a crisis more easily 

• Avoid conflict cycle 

• Understand kids’ reactions to situations better 

E. Other 

• Help parents to see what level their child functions on in the different domains 

• Supervise staff more effectively 

• Reinforce to staff the use of the DTORF 

• Feel supported in a strength-based approach to kids 

• Refresher course in restraint was helpful (?) 

• Only had the beginning class so it was not informative to me 

• See use and benefit [of DT-DT] for adolescents 



Appendix I (continued) 

What were the strengths of the training you received? ( n = 21 ) 



A. Material/Content 

• Overview of DTORF 

• visual of the “tree" 

• themes, devising a lesson for a particular stage 

• learning strategies to keep classroom safe and function in a positive healthy manner 

• learning a universal assessment system 

• learning some new strategies 

• Good overview 

• [received] help with lesson plans 

• having levels explained for grouping students 

• clearly defining growth in each stage necessary to cope with everyday life 

• positive, structured material 

• emphasis on conflict cycle 

• good notebooks and overheads, quality of materials 

• applicability to SEED population 

• comprehensiveness 

• seeing where my kids were developmentally and how I needed to change the curriculum to 

meet their needs 

• DTORF 

B. Organization/presentation 

• (DTORF) training well organized 

• practice sessions for the DTORF 

• informative and useful workshops 

• small group exercises 

• relevant examples 

• chance to discuss specific cases 

• role playing LSCI 

• comments/feedback from trainer 

• enjoyable sessions 

C. Trainers/mentors 

• supportive nature of the trainers 

• knowledgeable trainer 

• very personal and courteous instructors 

• help-fill input on individual situations 

• presenters offered information on an understandable level, good examples and moral 

support 

• helped with application 

• trainer there to help - nonjudgmental 

• trainer was extremely help-ful, not Just with ideas and theory - was involved with making 

concrete, speci-fic changes in classroom. Role-modeled interventions. Could put 
her work into action. 
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Appendix I (continued) 

What M/cre the weaknesses of the training? (Ni=20) 



A. Length of training and timing of training 

• time-slots of observations 

• need more time 

• meetings lasting longer than 3:30 p.m. (Dismissal time) 

• great deal of information packed into a short learning period (memorization for the sake 

of passing an exam.) 

• training was after school when we were tired. It was hard to pay attention 

• little chance for observations and feedback not due to the team’s inefficiency; poor 

coordination of visitation from our district. 

• Not enough time to cover everything we need help on. Sometimes I feel rushed in these 
workshops 

• Never enough time 

• too much info - too little time to absorb 

• Not enough time with [the trainer]. My classroom joined the project late so that may 

have been part of the problem. 

B. Content of training 

• Feedback to staff not directive enough 

• Few specific curriculum/activity ideas 

• would have liked more time for DTORF case studies and practice 

• not enough follow-up and practicing what was learned 

• application in the classroom to write goals and objectives 

• good suggestions but not enough follow-through 

• would like to observe “pros” at work actually implementing in our setting. Does not 

really satisfy new IDEA ‘97 requirements for lEPs or provide adequate goals for alternate 
assessment. You’ve tried to objectify but they aren’t really measurable in many instances 

• Comprehensiveness. Requires in-depth training almost above and beyond that possible 
when put into relationship with other requirements.. 

• Not enough classroom feedback, not enough observation. Our program does not debrief 

the way we should. Also, the training should put more emphasis on non- 
confrontation [al] verbal techniques and body language. I see this as a problem in 
new staff and they need to hear this in the program. More emphasis is needed on 
reflection, interpretation and redirection. I also was surprised, belatedly, that the 
session was over. Thought we had so much more to learn that I wondered if a 
problem had occurred that hastened our training to its end. I could have used a 
lot more workshop or observation time. From my point of view, staff needs a lot 
more training in doing a DTORF, selecting materials, doing theme based units and 
feedback. 



C. Whoops... 

• hard to get a return call or email quickly if we really needed help 
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Appendix I (continued) 



Have, you notice positive changes in your effect on children and families 

as a result of the training? (N= 2 1) 



A. YES 

• Yes. (5) Definitely. (1) 

• I have noticed a more improved program focus with common language being spoken on a 
daily basis. 

• Some parent are becoming more realistic about their child's functioning compared to 
typical peers. 

• We were able to control the behavior problems much easier. We also understood that 
they may be 8 years old but their stage says “they’re only 2;” therefore, their behavior 
may be normal. 

• The staff involved are better able to plan for their students. 

• Some of my groups are more effective - some children who are placed with their 
age/grade group are way behind their peers in their psychosocial development. 

• I can see us promoting appropriate student interaction according to the students’ 
developmental stage. 

• I believe using the DTORF as an evaluation tool for the child has aided in parental 
awareness of the child’s strengths and weaknesses. 

• I am trying to develop a more tolerant attitude in working with kids and families. 

• Positive on £r with the kids. The weekly meetings are helpful to brainstorm /reflect. 

• yes, but it is up and down. 

• yes, in students. Reaction from parents - not sure, not really observable, but in students 
there is more trust, more information shared and students in some cases are supporting 
each other more often. 

• I have seen indications that students may see a difference in staff. 

• I’m not sure I would have made it through the year had it not been for [my trainer] and 
the project. Even though it add some additional demands - the payoff was well worth it! 



B. No 

• Not yet (2) 

• I am no longer working with children I completed training with. 



What changes would you recommend in our future training for others? (N=l8) 

A. Workshop/introductory training (n=18) 

• More time for workshops 

• Present information more in “laymen" terms for parapros. 

• Prior to doing a workshop, do a survey to determine the needs of the group so that you 
could include this in the planning. 

• Workshops on teacher workdays, not after school. 

• Encourage [participants] to match information with relevant examples of student 
behavior 

• More small group exercises emphasizing the developmental task of each stage. 
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Appendix I (continued) 



• A general overview of Developmental Therapy at each training 

• Providing the book and materials ahead of time and asking [participants] to read first. 
Without a psychological background many of the concepts are new to them. We have a 
lot of provisional teachers comingfrom different educational fields, e.g., criminal justice, 
counseling, etc. 

• In trying to get teachers and other staff at a large program such as ours to buy into a 
“new" philosophy and way of relating to students, it would have been helpful for all staff 
to have had a brief hand-out about DTORF to read before the first training session 
(applies to both school years, as some staff were new to us this year.) 

• a video of good developmental teaching of stage I and II kids with PDD/Autism would be 
wonderful. 

• More concrete suggestions for ideas to implement in the classroom. 

B. Extended training during school year 

• a modified instructional delivery for staff training. Material is comprehensive and 
requires a slow, in-depth approach for staff to “buy-into” and put into action. 

• Involve more classrooms and teachers - maybe even create an entire building with the 
levels as the programs and acquiring the skills to move up, then out and back to public 
school or community when [the student] is developmentally appropriate for grade/age. 

• One-on-one help in the classroom to get these programs “user-friendly” - confidence to 
instit te and pass on to new staff the programs we have been mandated to use. 

• More support to help implement program in my classroom. 

• Spend more time with the teachers - when you meet with them after an observation 

• More observations with direct feedback.. 

• More consultation on real life implementation. 

C. Other 

• I would love to see this as coursework offered to teachers. I think the information would 
prove valuable for any type of classroom 

• Definitely update the DTORF notebook fold out with examples of skills. The are all, or 
seem to be, from children. It is sometimes difficult to assess adolescents by the present 
examples. The videos were good. I would add more good examples and contrast them(a 
few) with bad examples. 

• none (2) 

• Send out [evaluations] right after our training so I can remember better. 
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